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Abstract 

This report describes a computer system that creates simple computer animation in 
response to high-level, vague, and incomplete descriptions of films. It makes its films by 
collecting and evaluating suggestions from several different bodies of knowledge. The 
order in which it makes its choices is influenced by the focus of the film. Difficult 
/""*% choices are postponed to be resumed when more of the film has been determined. The 

system was implemented in an object-oriented language based upon computational 
entities called "actors". The goal behind the construction of the system is that, 
whenever faced with a choice, it should sensibly choose between alternatives based 
upon the description of the film and as much general knowledge as possible. The system 
is presented as a computational model of creativity and aesthetics. 
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A Guide for the Reading of this Report 

This report is intended primarily to be read by Al researchers and students, 
i hope, though, that computer scientists, especially those in computer graphics, 
would read parts, that animators would look at it (at least the corners), and that 
those interested in creativity and questions in aesthetics and the theory of art 
might get something out of it. This guide indicates what sections are most 
appropriate for each of these groups. 

If you want to skim off the top-level ideas while avoiding the technical parts 
then, keep to the first, eighth, and eleventh chapters. If you are interested in this 
thesis for its unusual perspective upon questions relating to creativity and 
aesthetics then, read the first chapter, sections D through H of the tenth chapter, 
the last chapter, and the final appendix. If you are reading this primarily for its 
relevance to computer graphics, then read the seventh appendix for nice example 
of what knowledge-based graphics is all about, then read the first and eighth 
chapters, section H of the tenth chapter, and the second and sixth appendices. 
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Chapter I Introduction 

This thesis is about the creation of computer animation by a computer system in 
response to high-level, vague, and incomplete descriptions of films. The computer 
system is named Ani. 1 Ani was tested with a simple version of the story of 
Cinderella. The films produced are graphically simple; the essence of the films is 
the dynamics of their images. In making the films Ani chooses between several 
alternatives by gathering and evaluating many suggestions from different bodies of 
knowledge. 

Ani is a simple embodiment of the aesthetic principles that the arbitrariness 
of any choice should be minimal and that the coherence of a set of choices should 
be maximal. Arbitrary choices are avoided by the use of knowledge. Coherence 
demands that the different parts and aspects of the films have the proper 
inter-relationships. The coherence of Ani's films results from a control structure 
that postpones troublesome choices and focuses on the more important aspects of 
the films. The quality of Ani's reasoning in making "aesthetic" decisions is a 
reflection of the amount of relevant knowledge that Ani brings to bear and the 
reasonableness of its use. 



I. Ani is a nickname animators have for animation. 
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Introduction 

Many Al (artificial intelligence) researchers try to understand the processes 
of intelligence by building computer models. They have concentrated upon certain 
aspects of intelligence, primarily problem solving and perception. Creativity and 
aesthetics are a departure from the aspects of intelligence usually explored. 

Of course, many problem-solving Al programs create solutions to problems 
(occasionally even aesthetic solutions) and vision systems create descriptions of 
scenes. This research differs from these systems primarily in the way descriptions 
are created and constrained. Ani initially is faced with few hard and fast constraints 
and spends much of time generating new constraints. In contrast, the products of 
most Al systems are strongly constrained by the real world or by utility or 
efficiency considerations. A symbolic description of a scene produced by a vision 
system is expected to correspond to the real scene; a program written by an 
automatic programmer is expected to meet precise problem specifications; a story 
understanding system is expected to make some of the common sense inferences 
that a human might make. 

If the vision system instead produced a critique of the scene, or if the 
automatic programmer decided what its programs would do, or the story 
understander started editing the stories given it, then they would be in a situation 
similar to Ani's. These problems appear to be too difficult to tackle currently; Ani 
is just a small step towards the solution of one problem of this type. 
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Introduction 

Research on this unusual kind of problem lead to some development of Al 
techniques and concepts. For example, Ani is built upon the idea of suggestions 
which are gathered, combined, elaborated, compared, related, compromised, 
rejected, and followed. Ani's control structure is somewhat unusual because it 
decides which choice to think about next and when to postpone decisions. 
Whenever a choice is difficult to make for reasons such as insufficient information 
or conflicting suggestions, Ani considers postponing that choice until more of the film 
has been determined. This postponement mechanism depends heavily upon Ani's 
explicit reasoning and recording of justifications. Ani uses a focus, which describes 
what parts of the film are to be emphasized, as a guide in making decisions. 

A computational or Al view of many of the classical problems in psychology, 
linguistics, and philosophy has shed some light upon questions such as "What is a 
concept?", "How do we see?", and "How does one learn?" and many more detailed 
questions by considering how intelligent mechanisms might be built. This thesis 
attempts to shed a little of this sort of light upon questions such as "How does an 
artist create?", "What is an art object?". Admittedly, the role of knowledge in Ani's 
making of aesthetic choices is very crude and simplistic in comparison to its role in 
human creativity. On the other hand, one can learn much from a relatively simple 
model of a complex process because of its very simplicity. 



Page - 1 3 



f\ 



What Ani Does Introduction 

Section A What Ani Does 

Animation is an anachronism. It is that rarity - a handmade product in a 
mechanized age. 

Marc Davis, a directing animator for Disney's Cinderella [Thomas 1958] page 134 ■ 

Ani is a computer system which, when presented with a description of a film, 
attempts to create an animated film based upon that description. A user presents 
Ani with partial descriptions of the personality and appearance of the characters 
involved, of the relationships and interactions among the characters, and of the type 
of film desired. Ani integrates this information with more general knowledge and 
jf**^ produces a detailed film description. This detailed description is a program that is 

run by a general purpose computer graphics system called Director [Kahn 1978b]. 
These programs written by Ani are at the level of detail of most computer graphics 
programs. 1 Ironically, a user can communicate with Ani only at a very high level. 
There is currently no mechanism for indicating to Ani how something should move or 
where it should be. 2 



¥ \ 



1. Examples of such programs are in Appendix VI. "Some Code Written by Ani" on page 272. 

2. Because Ani is lacking in an ability to handle appearances, currently one can describe an 
appearance onjv at a low level. 
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Introduction What Ani Does 

Ani was tested on a simplified version of the story of Cinderella. The events 
of the story are highly abstracted in keeping with the abstract objected-oriented 
dramatic animation that Ani makes. The abstraction frees Ani from needing to know 
the enormous number of facts required to animate "real world" detailed events (for 
example, those needed to animate Cinderella putting on a glass slipper). Ani cannot 
accept descriptions in English. Instead the film descriptions are presented to Ani as 
nested parenthetical expressions. A paraphrasing of the film description of the 
Cinderella story presented to Ani follows. 1 

Cinderella is physically both beautiful and shabby. She is good, 
friendly, hard working, and shy- Her stepmother is ugly, mean, selfish, 
strong and evil. Her stepmother hates and dominates Cinderella, who 
is obedient and tolerant of her. The fairy godmother is pretty, magical, 
good, kind and strong. Cinderella is polite and grateful to her fairy 
godmother, who is protective of Cinderella and helpful. The Prince is 
handsome, strong, good, stubborn, and determined. Cinderella and the 
Prince love each other. 

In the first scene, which is longer than average, the personalities of 
Cinderella and her stepmother and their relationship are established. 
Both characters are slightly happy. The next scene is also long and in 
it is conveyed that Cinderella wants to meet the Prince, that her 



I. The film description, in the format presented to Ani is in Appendix I. "The Cinderella Film 

Description" on page 244. 
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stepmother prevents that meeting and then Cinderella becomes very 
unhappy while the stepmother becomes happy and proud. 1 In the next 
rather short scene the fairy godmother undoes whatever the 
stepmother did in the earlier scene that prevented Cinderella from 
meeting the Prince. The next scene is long and in it is conveyed that 
Cinderella and the Prince are alone, then that Cinderella "gets it on" 
with the Prince, 2 and finally their relationship is established (that they 
love each other) and both become very happy. In the last scene the 
stepmother becomes aware of this and becomes very unhappy. 

In addition, the film should be moderately varied, simple, very coherent, 
obvious, energetic, not very flashy or original and last between 2 and 
5 minutes. The film should focus on the characters, especially 
/"""N Cinderella. The scene in which Cinderella meets the Prince and the 

scene where she is kept from him should be focused upon. Also the 
relationship between Cinderella and her stepmother are part of the 
focus. 



1. The next scene was a long one where the fairy godmother caused Cinderella's appearance to be 
elegant and then they both become happy. It was removed since Ani does not currently handle 
appearances, much less changes in appearances. 

2. "Getting it on" is a generic term for communicating, dancing, walking together, etc.. 



Page 16 



/"N 



«A 



/*"*N 



Introduction " What Ani Does 

This is all the information presented to Ani that is special to the Cinderella 
film. The film can be seen by flipping the corners of this report starting in the 
upper left, then the lower left, upper right and finally the lower right. Your rate 
should ideally be about 4 pages per second. 1 The appearances of the characters 
are not currently determined by Ani. I choose the following shapes and sizes for 
them. 



Cinderella 



Stcpnirilhcr 



The Prince 



Fairy (Wlnnllier 
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Figure 1.1 The Cast of Characters 



I. A color version of this film (the colors were chosen by me) that consists of about 4000 frames 
(about three minutes) was also produced. 
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What Ani Does Introduction 

First the stepmother pushes and chases Cinderella around for a while. Then 
she hits her several times. The Prince walks near and Cinderella moves over to him 
and they walk across the screen together. A film cut occurs and Cinderella and the 
Prince are at opposite ends of the screen. The Prince wanders across the bottom 
of the screen as Cinderella moves towards him only to be continually blocked by 
her stepmother. The fairy godmother appears and accompanies Cinderella over to 
the Prince. The stepmother and fairy godmother walk off the screen and Cinderella 
and the Prince turn back and forth and then walk across the screen together. A film 
cut occurs and Cinderella and the Prince are at opposite ends of the screen. They 
move towards each other and walk together. The cut and moving together happens 
again but with different dynamics. Cinderella and the Prince move close and then 
apart slightly and repeat this several times while the stepmother moves over to 
them and turns towards each. These activities and the reason why they happen are 
described in Chapter II "The Making of the Cinderella Film" on page 62. 

Section B What Ani Does Not Do 

Ani animates only by choosing the locations and movements of the characters. An 
embarrassing limitation of Ani is that the Cinderella story is the only story that Ani 
can animate. The main reason other stories can't be animated is that Ani has very 
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incomplete bodies of knowledge. 1 The extent to which Ani can make a reasonable 
film for the Cinderella story depends strongly upon the usefulness of the 
descriptions of words such as "shy", "hates", "varied", or "prevents" for generating 
and making choices. There is no reason to think that similar words not currently 
known to Ani, such as "clever", "fears", "textured", or "helps" could not be added. 
Words that are not as similar to the kinds that Ani currently deals with might, 
however, require new mechanisms or an extension of the representation. 

The limited range or variety of films Ani can make is not solely due to a 
limited vocabulary of descriptors. Ani is limited to very abstract descriptions and 
cannot handle very concrete expressions such as "a glass slipper", "runs down the 
stairs", or "a pumpkin turns into carriage". These would require an ability to draw 
as well as much more "real world" knowledge. The system is also limited 
stylistically to "object-oriented" films that are based upon stories. All these 
limitations are serious and are discussed at length in Chapter X "Improvements and 
Extensions" beginning on page 162. 



I. It seems th.it there is much room for Ani to grow within the current design to accommodate the 
knowledge needed to animate many other stories of the level of complexity of the Cinderella story. I 
doubt Ani can grow too large on simply a concept by concept basis. It is very unclear at what point 
increasing Ani's repertoire will necessitate much reorganization and rethinking. 
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How Alii Makes Films Introduction 

Section C How Ani Makes Films 

The reasoning Ani performs to produce the Cinderella film is lengthy and depends 
upon many different sources of information including the partially completed film 
description. The typical dynamics of each character must be determined in order to 
give them a convincing personality. Ani must decide which actions are most 
appropriate for the characters to perform and describe these actions in great detail. 
All of these choices need to be interwoven to produce a coherent whole that fulfills 
several global constraints. 

This section begins by describing one of the thousands of choices that Ani 
makes — at what speed should the stepmother typically move. One small piece of 
this example is then described in great detail. This is followed by a discussion of 
general mechanisms involved in the process. Finally a very different example of 
Anrs behavior — - the elaboration of the second part of the second scene — is 
presented. 

Choosing a Speed for the Stepmother 

One of Ani T s problems is to choose typical speeds for Cinderella, her stepmother, 
the fairy godmother, and the Prince which are in keeping with their personalities, 
physical characteristics, and the desired style of the film. The relative speeds of 
the characters should, in turn, be in keeping with the relationships and comparisons 
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of the characters. A simple example of aesthetic choice is the determination of 
these typical speeds. This choice is aesthetic because the character of movement 
is important in assessing the quality of animation, because it is interdependent with 
other choices, and because it is only weakly constrained by pragmatics. 

Choice points are created to represent the process of picking typical speeds 
for each character, The choice point for the stepmother's speed, for example, 
begins by asking each of the descriptors of the stepmother for suggestions for her 
speed. Only the description "strong" replies and suggests a high speed. The choice 
point is not happy with just that because there are not enough reliable suggestions. 
So it asks permission to be postponed to wait for more information to become 
/*""% available and it is granted. 

When the choice point for the stepmother's speed is reawakened, it inspects 
its record of previous activations. It then asks the choice points for the relative 
speeds of the stepmother and the other characters for suggestions. These choice 
points are created in response to this request and they choose values (e.g., that the 
stepmother be faster than Cinderella because she dominates Cinderella and differs 
from her), but cannot make any concrete suggestions since none of the characters 
have speeds yet. The choice point for the stepmother's speed asks permission to 
postpone to wait for the speeds of the others to be determined and it is granted. 
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Choosing- a Speed for the Stepmother Introduction 

The choice points for the other characters also ask and are granted 
permission to postpone. This could potentially lead to a deadlock in which the four 
choice points wait for each other to make a decision. One of the reasons the choice 
points don't just postpone themselves, but instead ask permission first, is to avoid 
this type of situation. A postponement manager keeps track of the situation and 
will not grant someone permission to postpone for the same reason twice. A 
common exception to this is when the choice point is waiting for other choice points 
to finish and at least one of these is making progress. In this case, no one is making 
progress so the postponement manager must refuse permission to at least one of 
the choice points. 

f **^ Ani is built upon the principle that as few decisions as possible be 

determined arbitrarily. The decision as to who should be refused permission to 
postpone has too many consequences to be determined by something like who asks 
first. Instead the postponement manager asks the focus, which indicates that 
conveying the personality of Cinderella is important. The choice point for 
Cinderella's speed is refused permission to postpone and the deadlock is broken. 
This means Cinderella's speed will be based on the description of Cinderella without 
being constrained to be faster or slower than the others. 
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Introduction Choosing a Speed for the Stepmother 

The choice point for the stepmother's speed finally gets suggestions from the 
relative choice points. It discovers conflicts with one of these suggestions and the 
earlier suggestion it had received from "powerful" and postpones again. Upon being 
resumed the choice point asks the descriptions of the film's style for suggestions 
and receives them from the moderate variety level, high energy level, and low 
flashiness. Unfortunately they do not all agree and so the choice point postpones 
one more time. 

When it is reawakened it discovers that there are no more sources of 
suggestions and proceeds with what it has. First it attempts to make compromises 
between the conflicting suggestions and makes one that in turn generates a new 
^""N conflict. Excuses are found for rejecting some of the conflicting suggestions. The 

choice point finally picks a high speed for the stepmother and saves away a 
justification for this choice. 

A Detailed Look at a Small Piece of Ani 

The program fragments soon to be introduced are written in Director, an actor 
language implemented by the author. [Kahn 1978b] The fundamental units of 
computation are actors. They are described elsewhere (e.g., [Hewitt 1975a], 
[Hewitt 1975b], [Hewitt 1977a], [Hewitt 1977b] and [Kahn 1978d]), but briefly 
they are independent modules containing both state and program that communicate 
by passing messages. In Director the state of an actor consists of its own variables 
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and a pattern-oriented database. The program part of an actor, usually called an 
actor's script, consists of an ordered list of methods. A method consists of a 
pattern and body describing the actions to take if the pattern is matched by an 
incoming message. Director's actors are embedded in a specialization-generalization 
hierarchy which provides inheritance of variables, database items, and methods. 

Consider the following paragraph from the previous section. 
Choice points are created to represent the process of picking typical 
$peecl$ for each character. The choice point for the stepmothers 
speed, for example, begins by asking each of the descriptors of the 
stepmother for suggestions for her speed. Only the description 
"strong" replies and suggests a high speed. The choice point is not 
happy with just that because there are not enough reliable 
suggestions. So it asks permission to be postponed to wait for more 
#*""**\ information to become available and it is granted... 

What do the sentences above mean? How does a "choice point" ask "each of 
the descriptors of the stepmother for suggestions for her speed"? How can one ask 
"permission to be postponed"? We shall present very detailed answers to these 
questions. 

First we will consider what the sentence "Choice points are created to 
represent the process of picking typical speeds for each character" means. It 
means that some actor in Ani sent to an actor named "absolute-choice-point" the 
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Introduction A Detailed Look at a Small Piece of Ani 

message "make (choice-point-of stepmother speed)" as follows,' 

(ASK nbsolute-choice-point MAKE (choice-point-of stepmother speed)) 

This creates an actor named "(choice-point-of stepmother speed)" which is 
an instance of "absolute-choice-point". This newly created actor just knows its 
task which is to choose a speed for the stepmother. When it cannot handle a 
message it will ask "absolute-choice-point", its parent, to handle it. 
"Absolute-choice-point" can handle a few trivial messages and otherwise passes 
them on along to "choice-point" who can handle about ten different messages 
ranging from requests for making a choice to receiving and combining groups of 
suggestions. "Choice-point" in turn passes those messages it cannot handle on up 
to "Something", a primitive actor in Director. The piece of the inheritance hierarchy 
involved in this example is depicted below. 



I. Director does not distinguish between upper and lower case. In the text that follows Director 
primitives are in upper case for expository purposes. 
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Some-thing 



V 

Choice-Point 



Rb^o I u+e -Cho i ce-Po i nt 




Introduction 
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Figure 1.2 A Piece of the Inheritance Hierarchy 
Some actor wants to know what the stepmother's speed is so it sends the 
new choice point a message asking it for its choice as follows. 
^ASK (choice-point-of stepmother speed) RECALL YOUR choice) 

A method for "recall your choice" is found in "choice-point". 1 The real method is 



1 If there wns no such method, the general method for messages matching "(recall your ?)" would 
hnve been found in the primitive actor named "Something" instead. 
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quite long and complex so what follows is a simplified version for exposition. The 

method was added to "choice-point" as follows. 1 

(ASK choice-point DO WHEN RECEIVING (recall your choice) 
; ; If the message "recall your choice" is sent, then 
(CONO ((ASK MYSELF RECALL YOUR current-choice)) 
; ; I answer with my current choice if I have one. 
(T ;; Otherwise I either have never tried to make this choice before 
; ; or I tried before but postponed for some reason. 
(LET ((postponement-reasons 

(ASK MYSELF RECALL YOUR postponement-reasons))) 
; ; So I recall what my reasons for previously postponing were. 
(COND ((NULL postponement-reasons) 

; ; There are no reasons so I've this is my first try 
(ASK MYSELF combine suggestions 

; ; I combine the suggestions I get by 

!,(ASK MYSELF collect suggestions from 

; ; asking myself to collect suggestions from 
; ; the first of my suggestion sources 
.(FIRST 

(ASK MYSELF 

RECALL YOUR suggestion-sources))))) 
(T ;; If I've previously postponed work on my choice 
(ASK MYSELF continue to recall your choice 
; ; this part is really much more complicated 
despite , (FIRST postponement-reasons)))))))) 

The running of this method produces the following series of transmissions. 



1. If seeing lots of programs, especially in a strange language, disturbs you. take heart in the fact that 
only this section of this chapter has much. 
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; ; First '-<?f if a value is already chosen and if so it is returned 

(ASK (choice-point-of stepmother speed) RECALL YOUR current-choice) 

NIL ; ; NIL is returned indicating- no choice has been made yet 

; ; Np\t see if a previous attempt to choose a value temporarily gave up 

(ASK (choice-point-of stepmother speed) RECALL YOUR postponement-reasons) 

NIL ; ; The NIL returned means that this is first time it has been asked for a value. 

; ; Now see what the possible sources of suggestions are. 

(ask (choice-point-of stepmother speed) recall your suggestion-sources) 
: ; The follov mc is returned after being found in "absolute-choice point" 
((absolute-suggestions ;; These first three sources are grouped together 

noighbors-absolute-suggestions ;; indicating that they should be explored together, 
opposites- absolute-suggest ions) 
relative- suggest ions 
global -suggestions) 
(ASK (choice-point-of stepmother speed) 
/*•"■■% collect suggestions from (absolute-suggestions 

neighbors-absolute-suggestions 
opposites-abso lute-suggest ions)) 

The last transmission invokes a method search which finds the following 

(again "doctored") method in "choice-point". 

(ASK choice-point DO WHEN RECEIVING 

(collect suggestions from ( ?f irst-suggestion-source %rest-of-the-sources)) 
When I get a message asking me to collect suggestions from a list of sources 
I gather up the suggestions from the first of the sources 
and combine them with those from the rest of the sources 
(APPEND 
(ASK MYSELF collect suggestions from , first-suggestion-source) 
(ASK MYSELF collect suggestions from ,rest-of-the-sources)) ) 

This method is placed near the end of the list of the methods in "choice-point". 
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Before it are methods for actually collecting suggestions from a single source. An 

example is the following one for collecting what are called "absolute-suggestions". 

(ASK choice-point 

DO WHEN RECEIVING (collect suggestions for absolute-suggestions) 
; ; when I get a message asking for absolute suggestions 
(ASK MYSELF 

ASK YOUR thing ;; i ask my thing, i.e. the object I am making some choice about 
; ; to collect suggestions for the element or aspect of my thing that 1 am choosing 
collect suggestions for ,(ASK MYSELF RECALL YOUR element))) 

This method causes a message to be sent to the "stepmother" asking her to 
collect suggestions for her speed as follows. 
(ASK stepmother collect suggestions for speed) 

/* The method invoked by this transmission had originally been added to an actor 

named "character", the stepmother's parent, as follows. 
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(ASK character DO WHEN RECEIVING (collect suggestions for ?element) 
; ; v.hm I get a message asking for suggestions for an aspect of myself 
(ASK MYSELF RECALL EACH ITEH HATCHING 

; ; then for every item in my database that matches the following pattern 
(description type ? ;; any type of descriptor is fine 
descriptor ?the-descriptor 
; ; call the descriptor "the descriptor" 
source ?.) ; ; any source 
THEN 

(ASK ,t.he-descriptor COLLECT ITEMS MEMORIZED MATCHING 
; ; then 1 ask the descriptor to search its database for items 
; ; matching the following pattern 
(suggestion element .element 

; ; a suggestion whose element is what we are looking for 
value ? 
strength ? 
source ?)))) 

We are now at the point corresponding to the second sentence in the 
paragraph which is "The choice point for the stepmother's speed, for example, 
begins by asking each of the descriptors of the stepmother for suggestions for her 
speed". The method above initiates the following transmission 
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(ASK stepmother RECALL EACH ITEM MATCHING 
(description type ? 

descriptor ?the-descriptor 
source ?) 
THEN 

(ASK ,the-descriptor COLLECT ITEMS MEMORIZED MATCHING 
(suggestion element speed 
value ? 
strength ? 
source ?))) 



/— N 



Which in turn generates the following transmissions. 

(ASK ugly COLLECT ITEMS MEMORIZED MATCHING ;; ugly is a descriptor of the stepmother 
(suggestion element speed 

value ? 

strength ? 

source ?)) 
NIL ; ; "Ugly" has no suggestions for speed 



^tfi**V 



(ASK mean COLLECT ITEMS MEMORIZED MATCHING 
(suggestion element speed 

value ? 

strength ? 

source ?)) 
NIL ; ; Neither does "mean" 
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(ASK strong COLLECT ITEMS MEMORIZED MATCHING 
(suggestion element speed 
value ? 
strength ? 
source ?)) 
; ; It turn? out that only "strong-" has a suggestion, the following one 
(suggestion element speed 
value high 
strength medium 
source strong) 

(ASK evil COLLECT ITEMS MEMORIZED MATCHING 
(suggestion element speed 

value ? 

strength ? 

source ?)) 
NIL ; ; "Evil" has no suggestions for speed 

(ASK selfish COLLECT ITEMS MEMORIZED MATCHING 
(suggestion element speed 

value ? 

strength ? 

source ?)) 
NIL ; ; Neither does "selfish" 



We have just witnessed what was described in the third sentence, "Only the 
description "strong" replies and suggests a high speed". 

This suggestion from "strong" is then returned by the "stepmother" and the 
message "(collect suggestions for absolute-suggestions)" previously sent to 
"(choice-point-of stepmother speed)" also returns the suggestion from "strong". 
There are still two other suggestion sources waiting to be tapped 
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"neighbors-absolute-suggestions" and "opposites-absolute-suggestions". They refer 
to suggestions from the antonyms and synonyms of the descriptors of the 
stepmother. The methods for collecting these suggestions are similar to the ones 
just described. In this case they return no suggestions at all. 

The suggestions gathered are then combined with any other suggestions 
previously gathered. In this case there is only the one from "strong", so no conflicts 
are looked for, no compromises sought, or problems postponed. 1 This suggestion 
from "strong" is added to the "(choice-point-of stepmother speed)". The 
"suggestion-sources" of the "(choice-point-of stepmother speed)" is set to those 
sources left, i.e. "{relative-suggestions global-suggestions)". 

We are about to see what was described in the last two sentences of the 
paragraph. "The choice point is not happy with just that because there are not 
enough reliable suggestions. So it asks permission to be postponed to wait for 
more information to become available and it is granted." An actor called "postponer" 
is then consulted as to what to do. 



1. "Combining" is a complex process that would at least double the length of the section if indudrd. It 
is described in Section D "Combining Value Suggestions" on page 95. 
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(ASK postponer should (choice-point-of stepmother speed) 
postpone with ;; the following suggestions 
((suggestion element speed 
value high 
strength medium 

source strong)) ;; and the following number of conflicts 
conflicts) 
; ; The answer from "postponer" to postpone follows, 
(postponed not-happy-enough) 

"Postponer" compared the suggestion in the message with the "minimally good 
sample suggestion" of the "(choice-point-of stepmother speed)" (which it inherited 
from "absolute-choice-point" and was set initially by the "focus"). The comparison 
in this case indicates that the suggestion from "strong" is not good enough and so 
"postponer" says to postpone. 

The "(choice-point-of stepmother speed)" then remembers the reason. 

(ask (choice-point-of stepmother speed) 

add not-happy-enough to your list of postponement-reasons regardless) 

At this point the choice point has altered its internal state so that upon 
reawakening it will know why it postponed previously, what sources of suggestions 
are left, which ones have yielded suggestions, what the suggestions have been 
gathered so far are, and what conflicts have yet to be resolved. 
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The choice point finally returns "(postponed not-happy-enough)" in response 
to the original message "choose a value". By doing this it has indicated to the actor 
that originally asked it for its choice that it has postponed its choice until later {and 
why). This actor can itself try to postpone to wait for "(choice-point-of stepmother 
speed)" or can decide to go ahead without knowing how "(choice-point-of 
stepmother speed)" will decide. 

The explicit form of "co-routining" between choice points has proved to be 
very flexible and general. It is up to each actor to obtain permission to postpone 
and to save away what it wants for its reawakening. A more conventional 
co-routining mechanism can be difficult to control and reason about. The major 
* N ' disadvantage of the kind of explicit co-routining that goes on in Ani is that any actor 

that uses it must have code that decides to postpone, that records some state 
information, and inspects such information upon reawakening. The inheritanc of 
methods and state from more generic actors greatly reduces this burden on an 
implementor however. 

This long description of just the first attempt (out of ten) to choose a typical 
speed for the stepmother is still sketchy. I hope it has served its purpose of 
conveying just what really happens inside of Ani and making it obvious why this 
level of description is confined to just this one section of a chapter in this report. 
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The Mechanisms Involved 

The example of the choice of a typical speed for the stepmother involved most of 
Anrs mechanisms for making choices. The general procedure begins with the 
exploration of a set of related choices represented as choice points. Each choice 
point starts by gathering up suggestions by asking the elements of its "choice" for 
suggestions. The choice points evaluate the suggestions by combining closely 
related ones, classifying any conflicts, and noting any missing information. The 
choice points which run into trouble ask permission to postpone themselves until 
more has been decided, while the satisfied ones make their choices based on the 
suggestions gathered. Permission to postpone is granted depending upon the 
reason for desiring postponement, the state of other choice points, and whether the 
choice in question is part of the focus of the object being created The intent is 
that when the choice point is awakened more information (or constraints) might be 
available making the choice easier. If permission to postpone is refused, more 
effort is expended despite the difficulty (e.g., more suggestions are gathered or 
conflicts are resolved), if no more progress is possible then a choice is made and 
its justification is recorded. 
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Suggestions are like rules, advice, or hypotheses that are rejected, modified, 
combined, and compromised. Suggestions differ from facts in that they typically 
conflict among themselves, are rejected or compromised, and possess degrees of 
reliability or strength. Suggestions are often ignored or modified with minor 
consequences. Ani is constantly faced with contradictory suggestions and spends 
much time detecting, classifying, and resolving these conflicts. Conflicts are 
resolved by making compromises and by rejecting some of the suggestions. 

A choice point represents the exploration of a choice. Each choice point 
responds to messages asking it to accept suggestions, to combine suggestions, and 
to make choices. A choice point maintains in its database records of the best 
suggestions so far, conflicts between suggestions, postponements, and the sources 
of suggestions that have already been tapped. Choice points decide whether a set 
of suggestions is adequate, more work needs to be done, or permission to postpone 
should be sought. 

If a choice point finds no conflicting suggestions for a value, the decision is 
simple and it just picks the value suggested. More typically there are many 
suggestions and they don't all agree. It is important that these conflicts be resolved 
as sensibly as possible if there is to be any coherence. The general approach is to 
use the heuristic with the strongest criterion of applicability. 
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The criteria for making a choice between two conflicting suggestions are 

(a) the strength of the suggestions, 

(b) the degree of compatibility with the other suggestions, 

(c) the extent to which the sources of the conflicting suggestions have 
held other suggestions followed, and 

(d) the number and kind of sources of the suggestions. 

If the difference between the conflicting suggestions is great along any of these 
dimensions, the rule associated with the criterion is used. 

Postponement is an important component in the making of choices that are 
interdependent. One choice adds new constraints that strongly influence later 
choices. Aesthetics only emerges when there is a set of inter-related choices; 
one hesitates to call an isolated atomic choice aesthetic. Because of this the 
order in which subproblems are attacked is very important. We want the 
relatively straight-forward choices to be made first because they already have a 
strong consistent justification. We want the choices with the least basis for a 
decision to be postponed as long as possible in the hope that by the time a 
choice finally has to be made additional constraints will have been added. 
Otherwise the choice has to be made on a relatively arbitrary basis and if 
chosen badly will cause trouble. Additional constraints often make the choice 
between the conflicting alternatives easier. The result of this control strategy is 
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that the final product is more coherent and less arbitrary. 

The postponement mechanism is designed so that the choices with the most 
justification are made first, then those that are difficult due to conflicting 
suggestions, followed by those with the least to go on. The order in which 
postponed choice points are forced to continue is under-determined by these 
criteria. The unordered choices could be made in parallel, however the 
interdependencies between them are such that the order of execution could 
adversely affect some of the decisions. 

To help avoid this arbitrariness, we have a structure called the focus of 
the object being created. It describes the parts or aspects that are primary or 
deserve emphasis (e.g., Cinderella's personality, her relationship with her 
stepmother, the second scene and so on). Those choices that relate to the 
focus (e.g., Cinderella's speed) tend to be made first and so are more likely to 
be self-consistent and effective because they are not constrained by choices yet 
to be made. 

The goals of this process of choosing is that the arbitrariness of each 
choice be minimal and the coherence of a set of choices be maximal. In Ani's 
filmmaking this means that each choice of any consequence should be compatible 
with the description of the film and Ani's general animation knowledge. 
Arbitrariness is minimized by use of knowledge in the form of suggestions and 
from the guidance of the focus. Coherence in general is a requirement upon the 
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internal structure and inter-relationships of the parts of the object. Coherence 
for Ani means that the choices for the relative dynamics of the characters be 
satisfied and that the choices of activities of the characters be self-consistent 
and compatible with the choices of the character dynamics. The aesthetics of 
Ani*s films are a result of this striving for coherence, this minimization of 
arbitrariness, and the currently small amount of knowledge about animation and 
emotions that Ani brings to bear. 

Examples of the Elaboration of a Scene 

To illustrate how a scene is elaborated let us consider how the second scene 
was animated. All that Ani is told is that it is long and in it should be 
conveyed that Cinderella wants to meet the Prince, that her stepmother 
prevents that meeting and then Cinderella becomes very unhappy while the 
stepmother becomes happy. Ani animates this scene as can be seen by flipping 
the lower right corner from page 85 to the end. First Cinderella and the Prince 
are placed on opposite sides of the screen. Cinderella moves a small way 
towards the Prince. Then the Prince begins to move in a slow arching manner 
along the bottom of the screen while Cinderella tries to move towards him. Her 
stepmother moves quickly from off the screen in front of Cinderella thereby 
preventing her from moving towards the Prince. The Prince continues to wander 
along the bottom while Cinderella repeatedly tries to move towards him only to 
be blocked by her stepmother each time. Towards the end of this activity 
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Cinderella's movements become more leaden while her stepmother's movements 
become more bouncy and spirited. 

It is hard to describe completely how Ani animates this scene because 
there are so many decisions and they are made at many different levels. Ani 
not only had to choose the activities, but also how the characters should move 
in them and how often sequences should be repeated. The initial locations of 
the characters also had to be chosen with care. Perhaps the best way to see 
how Ani made this scene is to just consider a few of the different types of 
decisions made. 1 

In the second part of this scene Ani needs to convey that Cinderella is 
/*""% prevented by her stepmother from meeting the Prince. First suggestions as to 

what should happen are gathered to provide a few alternatives to pick among. 
The actions are represented as actors 2 that can be asked for suggestions. In 
this case the main action is "prevents", so the actor for "prevents" is asked for 
suggestions. It suggests one of the prerequisites or enabling conditions of 
"meeting" be undone. The computation proceeds by asking "meets", the activity 



1. I confess that in what follows I probe into Ani's reasoning at the stronger or more interesting parts. 
Often Ani makes choices with little basis since because some aspect is underdeveloped. What follows is 
intended to give a good impression of Ani's decision making and knowledge base. 

2. Actors are computational entities that combine both program and data. They communicate bv 
sending and receiving messages. They are described in more detail in Section D "The Utility of Actors 
for Al" on page 44. 
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involved, for the prerequisites of Cinderella going to meet the Prince. It replies 
that Cinderella must be able to move ("can-move") to the vicinity of the Prince 
and that the Prince must initially be "on-stage". "Can-move" is then asked for 
suggestions for how the stepmother could make Cinderella unable to move to 
the Prince and replies that the stepmother can either "stand guard" (where the 
stepmother repeatedly blocks Cinderella from moving to the Prince), "make a 
fence" (where the stepmother moves leaving behind her a trail that Cinderella is 
unable to cross), or kill Cinderella. "On stage" is also asked for ways of undoing 
the fact that the Prince is on stage and suggests that the stepmother chase him 
off the screen. Ani's choice among the suggested activities depends upon how 
well their prerequisites are met, how well conditions which describe when the 
activities are most appropriate are satisfied, the strength with which the various 
activities were suggested, how well the activities can be run in the time 
allocated for this portion of the scene and how often each activity has been 
planned for other parts of the film. 1 

Ani chooses the activity of the stepmother standing guard between 
Cinderella and the Prince for this subscene. 2 Many more choices need to be 
made after deciding that this will happen. How often should the sequence of 



1. See Chapter II "The Making of the Cinderella Film" on page 62 for details on how this happens. 

2. Section B "The Kept Apart Scene" on page 65 provides more details and explains why this was 
chosen 
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Cinderella moving towards the Prince, the stepmother chasing, Cinderella giving 
up, and the stepmother relaxing be repeated? How fast should the stepmother 
move? How should Cinderella move? From where should the stepmother initially 
appear? 

Each of these choices is again based upon as much relevant knowledge as 
is feasible. Ani chooses a high speed for the stepmother because she is strong, 
faster than Cinderella, different from the fairy godmother (who has a medium 
speed), the desired level of energy of the film is high, and it is consistent with 
a moderate level of variety in the film. And yet Ani alters the stepmothers 
speed in this part of the second scene. When the stepmother is moving to get 
in front of Cinderella, she moves much faster than usual. Ani also increases 
Cinderella's speed in this scene but is careful that the stepmother is still faster. 
Even the normal speed increase for this activity is altered; it is lessened in 
order to stretch the running time of this subscene. 

By now the reader should have some feel for the kind of reasoning and 
knowledge involved in Ani's film making. My goal is that wherever you probe 
into Ani's decision making you find a few relevant aspects of the original film 
description combined in a reasonable way with a few general pieces of 
knowledge to make a sensible choice. 
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Section D The Utility of Actors for AI 

Hewitt ([Hewitt 1975a], [Hewitt 1975b], [Hewitt 1977a], and [Hewitt 1977b]) 
has developed a computational semantics based upon the concept of an actor. 
An actor is an entity that communicates with other actors by passing messages. 
An actor has its own state and this blurs the distinction between program and 
data. The idea is to give each actor the full capabilities of the digital computer 
that implements them. 

Ani was implemented in Director which is an actor-based language 
developed by the author {[Kahn 1976b] and [Kahn 1978b]) and as such provides 
evidence for the claim that actors are useful building blocks for implementing 
large AI systems. Inside Ani each character is an actor, as is every descriptor, 
character comparison, choice point, plan, method, scene, relationship and activity. 1 
The convenience of being able to place knowledge in an actor by adding items 
to the database associated with each actor and the power of being able to 
associate arbitrarily complex programs with the same actor were very important 
in easing the task of implementing and modifying Ani. Director's actors provide a 
means of chunking knowledge much in the way that frames do in FRL ([Roberts 
1977a] and [Roberts 1977b]) and units do in KRL [Bobrow 1977]. The use of 



1. Ideally everyt hing in the system should be an actor. In Director this would be too inefficient; in 
Innoun^es such as Smalltalk and Act I this is not the case. 
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actors eased the task of keeping the different components and bodies of 
knowledge of Ani as independent and modular as possible. Without this high 
degree of modularity, Ani would have been much more difficult to design, 
implement, and debug. 

Actors are important to AI not only as better building blocks for 
implementing intelligent systems. There is a way of thinking about how to 
organize complex systems that is sometimes called "the actor philosophy". It 
takes as the primary metaphor for modeling intelligence a community or society 
as opposed to the more traditional model as an individual. Knowledge and 
control is distributed to the extent feasible. This is not to suggest that it 
advocates anarchy over the commonly constructed hierarchical structure. There 
are many conventions and patterns in the way the components of a society 
interact. The implementation of Ani was strongly influenced by this view 
(perhaps not as much as it should have). For example, each source of 
suggestions is an independent module that puts forth its views of what should 
be done and in the case of conflicts with other sources becomes involved in the 
process of compromise or rejection that follows. The components of Ani are 
semi-autonomous but are not as independent as the subsocieties of agents in 
the "Society Theory of Mind" presented in [Minsky draft]. 
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Section E Ani as a Computational Model of Creativity 

// is irrelevant whether the system is running deterministically; what 
makes us call it a "choice maker" is whether we can identify with a 
high-level description of the process which takes place when the 
programs runs. On a low {machine language) level, the program looks 
like any other program; on a high (chunked) level, qualities such as 
"will", "intuition", "creativity", and "consciousness" can emerge. 

Douglas Hofstadter [Hofstadter 1979] page 71? 

A major contribution of this research to Al is a computational model of the 
design of an object. Much of what Ani does in making a film might be 
generalizable to the creation of such different objects as mathematical proofs, 
short stories, diagrams, or recipes. The creation of an object often begins with 
a description of a set of parts, some relationships between the parts, a temporal 
description of the interactions of the parts and some global constraints. It is 
very often useful to make comparisons between the parts to determine their 
degree of similarity and their level of uniqueness. The framework in which 
suggestions from these various components interact and are combined in Ani, 
should be applicable to many domains where one has ill-defined goals in a very 
large space of possibilities. The postponement of difficult choices and the 
guidance from a focus are also components of the creative process. 
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intimately connected to this model of creativity as a class of 
knowledge-oriented processes is a computational view of aesthetics. The two 
aspects which relate to aesthetics are Ani's avoidance of arbitrariness and 
striving for internal coherence in the films. Thoughts about how an "aesthetics 
expert" could be incorporated into Ani are presented in Section F "Incorporating 
a General Theory of Aesthetics" on page 194. Aesthetic issues are discussed in 
Section B "Ani and Aesthetics" on page 223. 

Section F Other Systems that Create Aesthetic Objects 

There are many computer systems that are claimed to create aesthetic objects. 
Few of these are based upon a knowledge intensive approach as is Ani. The 
two systems that I feel some kinship with Ani are Meehan's Talespin and Lenat's 
AM. They are discussed below. A discussion of other less closely related 
research can be found in Chapter VII "Other Research" on page 129. 

Talespin 

The problem in writing stories by computer is not bow to choose the 
level [of abstraction], but rather how to proceed once the level is 
chosen. 

-James Meehan [Meehan 1976] page 107 

One system that creates objects that are usually judged primarily on an 
aesthetic basis is Meehan's "Talespin" [Meehan 1976]. It makes up fables about 
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talking bears, birds, and so forth. Talespin is told the initial conditions (e.g., a 
bear is hungry and a bird in a tree is sleepy) and spins a tale based upon a 
high-level simulation. The characters of the story generate plans to satisfy their 
needs and desires. Executing these plans causes the characters to interact 
forming the substance of Talespin's stories. The resulting stories are plausible 
but typically not very interesting or aesthetic since there is no higher-level 
structure, to the stories and no notion of style or focus. Talespin has no sense 
of story aesthetics (e.g., a long story about a hungry bear trying to get food will 
end abruptly if it just happens across some food). Talespin is primarily a 
mechanism for producing realistic or plausible stories, not for creating literature. 1 
Meehan's research, despite the problem domain, is primarily concerned with plans 
and symbolic simulation, not with aesthetics. 

Talespin is often faced with aesthetic choices: what name to give the bear, 
where the crow should be, what kind of food should be available, and so on. 
These story aspects are aesthetic in our culture because they are aspects that 
we try to interpret as the result of purposeful choices made by the author. 
Not every aspect of a story is aesthetic. The number of letters in the name of 
a bear, the locations of the word "crow" on a printed page, and the amount of 
ink used in a description of food are not typically considered aesthetic choices. 



I. This point is clouded by a contemporary school of aesthetics that emphasizes a "slice of real life" 
style. Realistic stones, hyper realistic paintings, and cinema verite are all manifestations of this view. 
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Introduction Talespin 

And writers rarely make deliberate choices regarding these aspects. 

Talespin, however, rarely makes deliberate choices for those aspects that 
are normally considered aesthetic. Instead it either is told by the user, 
"chooses" randomly, chooses based upon a symbolic simulation of the characters 
involved, or chooses that which will help give the story a particular moral. The 
first two cases are not choice-making at all. The third is an interesting 
alternative to (or supplement of) the model presented herein. The difficulty is 
that simulation is concerned with plausibility, with having the components (the 
characters in the story) behave in a reasonable way. Aesthetics, on the other 
hand, is concerned with creating the world that such a simulation occurs in, with 
determining the rules of interaction, the goals of the components, and the initial 
conditions. The moral-fulfilling aspect of Talespin is more relevant to aesthetic 
choice. Unfortunately, it is a small, under-developed part of a large system. 

There are many differences between the Ani and Talespin. Ani must 
describe each character and each scene in great detail in order to actually 
animate them, while a story can leave out many details (and the story maker 
need never even think about them). Ani takes into account many aesthetic and 
stylistic parameters such as the level of variety, complexity, subtlety, flashiness, 
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Talespin Introduction 

coherence, and so on. 1 Ani can focus on those aspects declared to be 
important. Ani begins with a high-level description of a film, while Talespin 
starts with a description of the initial conditions and sometimes a moral that the 
story should have. Talespin simulates its characters, while Ani first arranges, 
predicts and plans things so that the subsequent character simulations fulfill 
higher-level goals. Subsection 1 "Relying Upon Knowledge Rather than 
Simulation" on page 205 contains a discussion of this aspect of Ani. 



I. To be fair. Ani's conception of these aesthetic and stylistic parameters is weak. The desired level of 
fhshiness, for example, influences Ani's decisions in only a few places. On the other hand, Talespin 
does not even attempt to take such factors into account. 
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Lenat's Artificial Mathematician 

Perhaps the greatest difference between AM and typical heuristic 
search procedures is that AM has no well-defined target concepts or 
target relationships. Rather, its "goal criterion" — its sole aim — is to 
maximize the interestingness level of the activities it performs, the 
priority ratings of the top tasks on the agenda. It doesn't matter 
precisely which definitions or conjectures AM discovers -- or misses 
--so long as it spends its time on plausible tasks. There is no fixed 
set of theorems that AM should discover, so AM is not a typical 
pro blem-solver . There is no fixed set of traps AM should avoid, no 
small set of legal moves, and no winning/losing behavior, so AM is not 
a typical game-player. 

Douglas Lenat [Lenat 1976] p. 9 

* y One of the more creative Al systems of late is Lenat's AM [Lenat 1976]. AM 

starts with very elementary concepts such as sets, composition, and equality. 
AM creates many new concepts, makes conjectures, and discovers new aspects 
of the original concepts. AM's heuristics propose tasks to perform that are 
explorations of this space and these are placed upon an agenda. The tasks on 
the agenda that are most interesting are performed first. 

AM is probably the research most related to the work described here. 
This may seem odd since AM's domain is elementary mathematics which is very 
formal and well understood -— almost the antithesis of aesthetics and art. 
Mathematics is formal and good models of it do exist, but, as Lenat points out, 
the exploration of mathematics, the heurstics that guide one in making 
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conjectures, in constructing new concepts, and in evaluating them, in other 
words, the doing of mathematics is neither formal nor well-understood. The 
problems that Ani and AM address are both weakly specified: making good 
animation and discovering interesting mathematics. Both systems construct 
structures out of a very large space of possibilities. Since the creations of AM 
and Ani are not judged as right or wrong — but as interesting or dull, 
plausible or implausible, good or bad — aesthetic choice plays a crucial role. 
Both AM and Ani are knowledge-oriented, in contrast with other approaches 
which are simulation-based, search-oriented, or based upon a few very genera! 
pieces of knowledge. Both systems spend a considerable portion of their time 
deciding what to do, in addition to doing it. 

There are many differences between the two systems, of course. The 
proposed tasks of AM can be viewed as suggestions from various heuristics as 
to what should be explored, however they are not treated as Ani's suggestions 
are (e.g., combining, compromising, relating, and so on). AM has a focus of 
attention which tends to keep AM from jumping from topic to topic. Ani's focus 
instead influences the relative priority of the elaboration of the different parts. 
The control structures of the two programs are very different. AM keeps 
executing the most interesting task on its agenda, while Ani jumps from choice 
to choice on the basis of their past difficulties and the focus. AM executes its 
tasks in a fairly straight-forward manner, while Ani works on a choice by 
gathering, combining, and rejecting suggestions, noting and classifying conflicts, 
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making compromises, and searching for more suggestions. 

A Digression into Aesthetics and Mathematics 

Popular views of mathematics, including the one that informs 
mathematical education in our schools, exaggerate its logical face and 
devalue all connections with everything else in human experience. By 
doing so, they fail to recognize the resonances between mathematics 
and total human being which are responsible for mathematical pleasure 
and beauty. 

Seymour Papert [Papert 1973] p. 104 

Many people would be surprised to hear that Ani and AM are so similar in spirit 
fm^ and purpose. They think that mathematics is formal, dry, precise, and iogical 

while animation (or art in general) is intuitive, informal, inexact, and emotional. 
There is some truth to this view of the products of the mathematics and art, 
but the differences nearly evaporate when considering the processes that makes 
them. Both art and mathematics are appreciated for their beauty. 1 More 
significantly both are partially evaluated on the basis of their structure and the 
quality of the thought or reasoning underlying their creation. Mathematicians 
often speak of what they do in terms such as beauty, creativity, aesthetics, and 
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I. It is an open question whether they are very different kinds of beauty, however. In Subsection I 
"What Aesthetics is" on page 223 I present a multi faceted view of aesthetics that is only partially 
applicable to mathematics. Mathematical aesthetics shares much with art aesthetics but differs in many 
significant ways suggesting that they might both be instances of some more general aesthetics. 
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intuition. In both domains one appreciates a complex product that is the result 
of simple means. 1 Mathematics (especially doing mathematics) is much more 
loose, intuitive, and emotional than is commonly thought. This thesis argues that 
art too is loose, intuitive, emotional and so forth, but that these aspects need 
not remain vague and mysterious but can be explicated, made more concrete, by 
building computational models of these processes. 



I. See .Section B "Aesthetic Systems" on page 133 for a discussion of this. 
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Section G Animation -- What is it? 

Animation is an animated film. 

A protest against the stationary condition. 

Animation transporting movement of nature 

directly cannot be creative animation. 

Animation is a technical process in which the 

final result must always be creative. 

TO animate: to give life and soul to a design, 

not through the copying but through the transformation of reality. 



Life is warmness. 
Warmness is movement 
Movement is life- 
Animation is giving life; it means giving warmth. 
/*""■% Animation could be tepid, warm or boiling. 

Cold animation is not animation. It is a still born child. 

Practically, animation is a long rubbing of tree against tree in order to get 

sparkle or perhaps a little smoke. 

Take one kilo of ideas (not too confused if possible), 

five dkg of talent, ten dkg of hard work and a few thousand designs. 

Shake it all together and if you are lucky 

you will not get the right answer to the question. 

Statement of several Yugoslavian animators 



IVs a way that I have found to make films where I can control 
everything. 

— Eliot Noyes. Jr. [Whitney Museum 1976] page S3 
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Animation What is it? Introduction 

To nwny, "animation" is synonymous with the cartoon — a narrative 
created by step-by-step drawing on and animation of cells, or cards... 
But, in fact, animation ranges from the well-known cartoon to abstract 
designs and collages. 

-• [Whitney Museum 1976] page 73 

Animation is the creation of an illusion of change and motion. Typically this 
illusion is accomplished by filming frame-by-frame successive drawings or 
positions of three-dimensional models. When the film is played back at a 
sufficiently high speed the discrete changes between frames are no longer 
perceived and images appears to move and change continuously. This differs 
from live-action film which attempts to reproduce changes in the real world. 1 

Animation is not limited to the medium of film. The nineteenth century saw 
the invention of zoetropes and other devices for creating the illusion of motion 
by quickly changing which drawing is visible. One very old medium for animation 
that is still in use today is a flip book of drawings which the viewer flips 
through very quickly. 2 Today some animation is produced solely with video 
equipment. A more esoteric animation medium consists of very quickly moving 
lasers that make animated drawings inside of planetariums. The animation 



1. Thr distinction dors not concern the images that make up the film, for an animated film can be 
constructed totally of real world images rather than drawn or constructed images. Such films are called 
pixilations and are animation because they create a synthetic motion. 

2. The pictures in the corner of this report are based upon the same idea. 
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discussed in this report is generated within a computer's memory and can be 
displayed by any of the animation media. Typically computer animation is 
displayed by some sort of computer graphic display (often a television set), 
though film, video, lasers, and even occasionally flip books are used. 

Animation has no limitations of style, content, or purpose other than those 
imposed by the imaginations and energy of animators. Cartoons drawn upon eels 
(layers of clear acetate) are the most prevalent, but animated films are 
produced that vary from being totally abstract dynamic images, to simple cartoon 
line drawings, to moving portraits. 

The kind of animation that Ani makes is unusual. Typically dramatic 
/**N animation uses figurative images to tell a story, while abstract animation seldom 

tells a story. There are a few films of non-figurative dramatic animation other 
than Ani"s. Most notable is a cartoon directed by Chuck Jones and Les Goldman 
calied "The Dot and the Line" which won an Academy Award in 1 965. Heider 
used a film consisting of two triangles, a circle, and a stationary rectangle in a 
series of psychological perception experiments [Heider 1944]. The film is very 
similar to those made by Ani. (Section A "The Kinds of Films Ani Makes" on 
page 72 discusses further the kind of animation Ani makes and why.) 



Page 57 



j0***\. 



/""""S 



/""N 



/**S 



Animation What is it? Introduction 

An animator creates a world, peoples it, defines its laws, and orchestrates 
its events. This view of animation is probably what Alan Kay, who among much 
else is an innovator in computer graphics, meant when he said that animation is 
simulation. The behavior of several interacting entities can be simulated by local 
computations. I agree fully that animation itself is most profitably thought of as 
simulation and in fact Ani simulates the film's characters. This thesis, however, 
argues strongly that the process of designing and conceiving animation is much 
more than simulation because of its use of high-level reasoning and knowledge. 
Ani simulates but, more importantly, carefully arranges the initial conditions and 
chooses the outcomes in probabilistic situations so that the desired events 
happen and at the desired times. To plan and arrange these things Ani often 
makes crude predictions such as how far a character will move before being 
stopped or how long an event will take. Animation is simulation; the creation of 
animation is problem solving. 
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Introduction Animation What is it? 

Section H Ani and Computer Graphics/Animation 

In the development of computer graphics, one very important area 
seems to be lagging the other areas rather markedly — and 
remarkably. It is the area that is inverse to abstraction. Let me call it 
"visualization". The basic notion is that the essential features of a 
complex structure or process can be abstracted from the residual 
detail and represented succinctly and economically for many uses, but, 
when it comes time for realistic presentation of the structure or 
process, the detail has to be put back in — or, as in an architectural 
drawing of a brick building, at least artfully suggested. Both the 
analytic process of abstraction and the synthetic process of 
disabstraction — visualization — are knowledge-based processes. In 
the field of artificial intelligence, a considerable amount of work is 
being done on abstraction. It is visualization that I think is being 
neglected. A natural home for work on visualization is computer 
.^^ graphics. 

J.C.R. Lickiider [Licklider 1976] page 9? 

Researchers in computer graphics and animation are in the business of creating 
and controlling images by computer. Their activities differ, however, from this 
research in that for the most part they keep the knowledge necessary to create 
and control images in their heads and do not explicitly put this knowledge into a 
computer. They program the computer explicitly by giving formulas and 
parameters and the like. Ani is better thought of as a bad imitation of such 
programmers than as one of their computer graphics programs. 
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Am and Computer Graphics/A mmatioi Introduction 

The explicitly programmed control of images by computer animators is very 
sensible — if you know what something should look like it is very much easier 
to describe to a computer the object than to describe how the computer might 
figure out what it should look like. I could have made Ani's Cinderella film 
myself in a few days, instead of the year it took to build Ani. Why should 
computer animators view Ani as anything more than a very crude, stupid, and 
limited imitation of themselves? Because, although it took more than a year to 
build Ani, it took less than an hour to describe the Cinderella film. 1 A poorly 
optimized Ani takes about an hour to produce the Cinderella film. Ani can make 
many variations of the film and, with a more complete knowledge base, could 
animate a large number of stories. Of at least as much relevance to computer 
graphics researchers is that this research makes explicit some of the knowledge 
that is involved in making computer animation. 

A major claim of this thesis is that Ani makes choices which is something 
that computer graphics programs do not currently do. People are often confused 
about this. Ani is a collection of explicit deterministic procedures as are all 
computer graphics programs — so how is Ani able to make choices? The 
distinction is subtle and depends upon different levels of description of 



i. Simplv describing a film to Ani does not take much time, but if Ani's knowledge base contains only 
a few of the descriptors or activities described then a good deal of time needs to be spent adding the 
missing,' ones. 
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Introduction Am and Computer Graphics/Animation 

programs. At one level of description both Ani and the usual computer 
animation program are diligently executing the instructions of the procedures that 

make them up Ani, however, can be sensibly described at a higher level in 

» 

terms of recognizing choices, gathering up relevant facts, applying imprecise and 
informal rules, making choices and so on. [Hofstadter 1979] and [McCarthy 
draft] both contain good discussions of this idea. 

Knowledge is both cumulative and useful in producing computer graphics. 
Knowledge is cumulative much the way subroutines are. It may be difficult to 
implement a particular subroutine but once done it is potentially usable in any 
program that needs it. Putting knowledge into a machine is difficult but, if done 
in a sufficiently general and modular way, is a very good investment of effort. 
That it is useful should go without saying were it not that it is rarely 
recognized in the computer graphics literature. Very rarely does any of this 
literature connect with work in knowledge-based programming and Al. 
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The Making of the Cinderella Film 

Chapter II The Making of the Cinderella Film 

This chapter describes the Cinderella film as Ani describes it to Director. Also 
presented is a high-level sketch of the process that scene by scene generated an 
elaborate description of the movie from a short and vague description. Alternatives 
that Ani considered and rejected are also discussed. An explanation of how the 
program described herein is executed can be found in Appendix II. "The Running of 
the Cinderella Film" on page 250. 

The segments of the film discussed can be viewed by flipping the corners of 
the pages indicated in the text. Cinderella is the star, the Prince is the triangle, the 
stepmother is the square, and the fairy godmother is the circle. 

Section A The Introduction Scene 

Ani was told that Cinderella is good, friendly, hard working, shy and has the most 
important role in the story, that her stepmother is ugly, mean, selfish, strong and 
evil, and that her stepmother hates and dominates Cinderella who is obedient and 
tolerant of her. In the first scene, the personalities of Cinderella and her 
stepmother and their relationship are established. This scene is longer than 
average and both characters are slightly happy. (It can be seen by flipping the 
upper right corner from page 3 to the end and the lower right corner from page 3 
to 83.) 
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Ani begins by assigning each of the six tasks (establishing the personality of 
Cinderella, the personality of the stepmother, the relationship of the stepmother to 
Cinderella, the relationship of Cinderella to her stepmother, the emotional state of 
Cinderella, and the emotional state of her stepmother) to a subscene. Ani works on 
each subscene separately gathering and combining suggestions. The subscenes are 
then sequenced, consolidated, eliminated, and reassigned resulting in the following 
four subscenes. 

(a) (This subscene can be seen by flipping the upper right corner from 
page 3 to 213.) To establish that Cinderella and the stepmother 
are slightly happy their dynamics are slowly changed so that their 
speed, acceleration, activity level, and rhythmic level are increased 
by about an eighth. While this is happening, in order to establish 
/•■v her relationship with Cinderella, the stepmother pushes Cinderella 

around. Ani repeats five times the cycle of the stepmother coming 
into contact with Cinderella, them moving off together (in the 
original direction), Cinderella continuing to coast and then moving 
away from the stepmother. During the entire activity the dynamics 
of Cinderella and her stepmother are altered by cutting in half the 
amount of curvature in their movements and by making sure that 
both characters move fast. For each component the dynamics of 
the characters are also modified. (For example, the first part has 
the stepmother moving 50% faster than normal when moving 
towards Cinderella and then they move together at 907. of the 
stepmother's normal speed.) Since this is the first part of the first 
scene Ani needs to give the characters initial locations. The 
stepmother is placed in the middle left part of the screen while 
Cinderella is placed a medium distance from her (and on the 
screen). 



A 
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Cinderella Film 



(I)) This section can be seen by flipping the upper right corner from 
page 215 to 219. To establish the personality of the stepmother 
as mean and evil Ani has her chase Cinderella around. During this 
subscene the stepmother's speed, acceleration and activity level 
are high and her rhythmic level is moderate while Cinderella's 
speed and acceleration are low and her rhythmic level and activity 
level are high. (This activity was poorly defined which accounts 
for its one or two second duration.) 

(c) This subscene can be seen by flipping the upper right corner from 
page 221 to 299. It also establishes the personality of the 
stepmother as mean and evil and does so by having her hurt 
Cinderella, in this case by going over to Cinderella and hitting her 
10 times. This choice was reinforced by being suggested also as a 
way of conveying that the stepmother hates Cinderella. 



f*\ 



(d) This section can be seen by flipping the lower right corner from 
page 3 to 83. To establish Cinderella's personality as friendly she 
goes over to the Prince and accompanies him across the screen. 
As usual, Ani modifies the dynamics of the characters as is 
appropriate for the activity (e.g. in the last part of this scene the 
Prince and Cinderella move according to Cinderella's dynamics which 
are slowed down 20% from normal). Also since the Prince is 
introduced in this subscene, his location is initialized as a medium 
distance from Cinderella and offstage (he moves on screen during 
this subscene). 1 



I If Ani were extended as described in Section D "Critics and Criticism" on page 18? then this would 
be .in idfnl test case, If we want Ani to stick a little closer to the original fairy tale (recall that Am is 
told a vcrv condensed and abstract version of the story of Cinderella and cannot be expected to follow 
the original story very faithfully without help), then we could criticize Ani's choices for this subscene. 
Wp could stmplv say that Cinderella is not to meet the Prince until the third scene and Ani would redo 
this subscene (and unless necessary no other parts of the film) picking something else to establish 
Cinderella's personality. 
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Cinclpj-fUa Film The Introduction Scene 

Ani decides to make the entire first scene take between 30 and 75 
seconds and estimates that it will take 43.3 seconds (which is surprisingly close 
to the 42 seconds it really took). These time constraints are one of the major 
factors in determining how many times the pushing and the hitting activities 
should be repeated. 

Holes in Ani's knowledge base turn up in running this scene and the user is 
warned that Ani has no idea how to establish that Cinderella is obedient 
towards and tolerant of her stepmother. The user is also warned that Ani had 
decided to have Cinderella help the Prince (to show that she was friendly) but 
didn't know enough to pull it off and canceled it. A similar thing happened with 
shy's suggestion that she avoid a stranger. 1 

Section B The Kept Apart Scene 

The "kept apart" scene is also long and in it is conveyed that Cinderella wants 
to meet the Prince, that her stepmother prevents that meeting and then 
Cinderella becomes very unhappy while the stepmother becomes happy and 
proud. To see this scene flip the lower right corner from page 85 to the end. 



1. There are a few reasons why I never plugged these holes in Ani's knowledge base. For one there 
are many and much time would be spent filling them in. For another it fits my perceptions of how an 
animator does, indeed, conceive of a film. Often an animator will think of a good idea and then reject 
it only because he or she does not know enough to carry it out fully. 
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The Kept Apart Scene Cinderella Film 

For this scene Ani creates the following three subscenes. 

(a) This section can be seen by flipping the lower right corner from 
page 85 to 91. To convey that Cinderella wants to meet the 
Prince, Ani moves her a small part of the distance towards him. In 
order that this be possible Cinderella and the Prince must be at 
least a medium distance apart and yet the last scene left them 
rip.ht next to each other. To resolve this a film cut is called for 
and new locations for all the characters are determined. The 
stepmother is no longer on stage and Cinderella and the Prince are 
placed at opposite ends of the screen. To help make this 
subscene more effective Cinderella's speed is set to a high value 
and she moves directly and deliberately. 

(b) This section can be seen by flipping the lower right corner from 
page 9] to 311. To convey that the stepmother prevents 
Cinderella from meeting the Prince, Ani repeats six times the cycle 

f\ of Cinderella moving towards the Prince, her stepmother blocking 

her, Cinderella moving towards the location that the Prince is 
headed only to be blocked again by her stepmother. 1 The Prince 
is set in motion just to make this activity more interesting (since 
he has nothing else to do in this scene). 

(c) This section can be seen by flipping the lower right corner from 
page 205 to 311. To establish that Cinderella becomes very 
unhappy she gradually slows down and moves in long straight lines 
rather than shorter curves. Concurrently, her stepmother speeds 
up and moves in long highly-arcing curves. This subscene has no 



I. Here is one of the many places where Ani would behave more sensibly were other parts of the film 
taken mto account. The previous subscene's effect is completely swamped by this one. The previous 
subscene is acceptable on its own. but considering that this subscene does its job even better it should 
either be eliminated or improved. 
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Cinderella Film The Kept Apart Scene 

activities of its own and so it asked the previous subscene to 
stretch itself and arranged to start its changes in the dynamics of 
the stepmother and Cinderella at the time the previous subscene 
estimated it would have stopped (had it not been stretched). The 
second subscene was stretched by both slowing down the action 
and by increasing the number of cycle repetitions (Ani originally 
planned to repeat the standing guard sequence only four times). 
The ability to stretch activities depends heavily on reasonable 
estimates for the running time of the methods. The methods 
involved in this subscene estimated that they would take 30 
seconds and they really took 32 seconds (which is closer than 
necessary). 

Various alternatives to the first two subscenes were considered and rejected. 
For example, the stepmother could have chased the Prince off the stage thereby 
preventing Cinderella from meeting him but this activity (as currently defined) 
cannot be stretched to last anywhere near as long as is required. The "stands 
guard" method was chosen over the "makes fence" method because of a better 
time fit (unlike the "chases off" method "makes fence" could run long enough). 
Besides having a better time fit the "stands guard" method was more successful 
in determining that its prerequisites were met than the "makes fence" method. 
Again holes in Ani's knowledge prevented some methods from being considered 
very fully (e.g., the stepmother killing Cinderella). 
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Section C The No Longer Kept Apart Scene 

in the next rather short scene the fairy godmother undoes whatever the 
stepmother did in the earlier scene that prevented Cinderella from meeting the 
Prince. 

This scene has only one subscene. It can be seen by flipping the upper 
left corner from page 4 to 38. Ani recalls that the previous scene used the 
"stands guard" method and asks that method for ways of undoing it. The 
suggestion that the fairy godmother chase the stepmother off the screen is 
rejected quickly because it would not take long enough. The fairy godmother 
could have killed the stepmother but, since Ani does not know how to display 
r^ this, it is dropped. One interesting suggestion is that the fairy godmother 

"recursively" stand guard between the stepmother and a place that is away from 
Cinderella in the direction of the Prince. This was rejected because the 
"standing guard" method requires that the guard be faster and dominate the 
"prisoner" which is not the case here. The method chosen was to have the 
fairy godmother accompany Cinderella to the Prince while the stepmother stands 
by helpless (i.e. she does nothing). The fairy godmother moves quickly and 
directly as she "drags" Cinderella along. 
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Cinderella Film The No Longer Kept Apart Scene 

This scene is a bit confusing to watch — things happen too quickly and 
the fairy godmother just appears and is quickly gone. User criticism could easily 
be applied to allocate more time for this scene or to have the fairy godmother 
start far from Cinderella. A discussion of how Ani could recognize and correct 
the situation are described in Section D "Critics and Criticism" on page 189. 

Section D The Meeting Scene 

The next scene is long. First it is conveyed that Cinderella and the Prince are 
alone, then that Cinderella "gets it on" with the Prince, and then their 
relationship (that they love each other) is established and both become very 
happy. Flip the upper left corner from page 40 to the end, then flip from the 
beginning to the lower left corner of page 96 to see this scene. 

Ani eventually breaks this scene down into the following seven subscenes. 

(a) To convey that Cinderella and the Prince are alone is easy — just 
arrange for all the other characters on stage to exit. This can be 
seen by flipping the upper left corner from page 40 to 62. 

(h) To convey that they are "getting it on" the sequence of Cinderella 
turning, followed by the Prince turning, Cinderella turning the other 
way and the Prince following suit is repeated three times. A 
prerequisite for this method (which is met) is that the two face 
each other and be ciose. This can be seen by flipping the upper 
left corner from page 64 to 218. 
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The Meeting Scene Cinderella Film 

(c) To help establish that Cinderella loves the prince she goes to him 
and accompanies him across the stage. This can be seen by 
flipping the upper left corner from page 220 to 290. 

(d) The next subscene can be seen by flipping the lower left corner 
from page 4 to 38. Another way to establish that Cinderella loves 
the Prince is to convey that she wants to "get it on" with him and 
to do this she does the "first part" of this activity, i.e. moves 
towards him. Cinderella moves at twice her normal speed and 
more deliberately, while the Prince moves to her at his normal 
pace (which is as fast as Cinderella in this episode). Since the 
previous subscene left the two lovers next to each other Ani 
makes a "cut", clears the screen and relocates the tw.o at opposite 
ends of the screen. 

(e) This subscene can be seen by flipping the lower left corner from 
page 40 to 74. To help establish that the Prince loves Cinderella 
he goes to her and accompanies her across the stage. This is the 

/*■"% same as the third subscene except the roles of Cinderella and the 

Prince are interchanged. 

(f ) This one be seen by flipping the lower left corner from page 76 to 
96. As with the fourth subscene, to help establish that the Prince 
loves Cinderella, he moves toward her to convey that he wants to 
"get it on". This time it is his dynamics that are altered and he 
moves very fast and directly. Again a cut is called for to position 
them far apart. 

(g) To show that they are both very happy, the dynamics of the Prince 
and Cinderella are changed. As in the last subscene of the "kept 
apart" scene, to carry out these dynamics changes the previous 
subscene is stretched and this one begins at the same time as the 
sixth subscene begins. 
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Section E The Justice Scene 

In the last scene the stepmother becomes aware of the Prince and Cinderella 
getting it on and becomes unhappy. It can be seen by flipping the lower left 
corner from page 98 to 258. 

Ani does not break this scene up. "Aware" suggests that the activity 
being observed (in this case Cinderella and the Prince getting it on) happen in 
parallel with the stepmother moving near them and slowly facing (looking) 
towards each of them in turn. The "getting it on" activity chosen is a repetition 
four times of the cycle of the Prince and Cinderella moving until they are in 
contact and then moving apart a little. While this is happening the stepmother's 
ifS movements slow down and become more leaden to convey that she is unhappy. 
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Chapter III Film Descriptions 

This chapter discusses the kinds of films that Ani makes and how films are 
described to Ani. The representation of a user's description of a film is briefly 
presented along with examples of its use in describing the Cinderella film. A 
discussion follows of what the words in the film descriptions mean in operational 
terms. Finally, small portions of a fully elaborated film description produced by Ani 
are presented. 

Section A The Kinds of Films Ani Makes 

Ani makes an unusual sort of animated film. They are intended to tell stories and 
yet the characters are not figurative and their behavior is limited to very abstract 
kinds of activities. Cinderella is a simple geometric shape that cannot even walk, 
much less dance at a ball or put on a glass slipper. Ani's films are not normal 
narrative animation, nor do they look like much of the abstract animation that is 
made (with or without the help of computers).' Why then was Ani designed to 
produce non-figurative dramatic abstract animation? 



I. A good description of abstract animation can be found in [Russett 1976]. 
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Figurative animation was rejected because of the difficulty of enabling a 
computer to draw in addition to animating. Producing a system capable of creating 
representational line drawings is at least as difficult as making Ani. Another reason 
I avoided figurative animation is because of the very large amount of real-world 
knowledge that is entailed if the appearance and behavior of the characters are not 
abstract. If the characters have faces then the viewers will expect them to smile 
when it is appropriate. If the characters have arms and legs then the viewers will 
expect them to walk, to raise their hands in greeting, to hold things etc. Placing a 
small number of such facts into the machine is not especially difficult [Kahn 1976a]. 1 
The difficult problems arise when one tries to manage the extremely large number 
of such facts about everyday life that are needed. This problem of putting common 
sense into a machine is an important topic of research in Al (see [Minsky 1975] and 
[Schank 1 977] for example). Some ideas as to how Ani might be extended to make 
figurative animation are presented in Subsection 6 "Figurative Animation" on page 
174 and in [Kahn 1976a]. 



t. An example of such a fact is that people typically raise their hands while greeting unless they are in 
a hurry or have their hands full. A different problem is knowing how exactly one moves one's arm 
while greeting or how to generate images of this activity. 
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The Kinds of Films Ani Makes Film Descriptions 

Non-narrative abstract animation was rejected as Ani's domain of expertise 
because such films typically are very perceptually-oriented. The computer need 
not see to make such films, only possess the knowledge that having eyes entails. It 
is less obvious, though, what that knowledge is like and how to represent it. Al 
research on vision is relevant here, but it has not advanced to the stage of being 
able to contribute to questions about perceptual aesthetics. Another difficulty with 
purely abstract animation is that it is much more subjective and harder to evaluate. 
Subsection 7 "Non-Narrative Animation" on page 176 discusses how Ani might be 
extended to produce this sort of animation. 

But if Ani makes films that are neither figurative nor abstract what is left? 
The essence of animation, dynamics or change, is what is left.' Eric Martin, an 
animator and teacher, has emphasized the role of dynamics in animation by 
statements like, "Animation displays process" and "movement has as much integrity 
as drawing". He has given class exercises to produce films with only one small dot 
that does not change color or shape or size. It is very instructive to see how much 
can be expressed this way. The dot's character can be conveyed as happy, 
burdened, neurotic, frightened or whatever. Certain areas of the space can be 
characterized simply by the way the dot behaves. If, for example, the dot avoids an 



I. It was probably wi<e to start with the essential aspects of animation, and consider Ani as a good 
base upon which to build more general animators. (See Section A "Making Ani Do More" on page 162 
for ideas on this.) 
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area, slowly approaches it and suddenly runs away, cne tends to think of the area 
as frightening or dangerous. Ani's replacement of dots with simple geometric 
shapes is unimportant and just aids the clarity slightly. 1 This type of film is not 
entirely novel. A notable example is Chuck Jones' Academy-award winning cartoon, 
"The Dot and the Line". 



It ain't the meat, it's the motion... 

- From a popular song by Maria Muldaur 

Section B Film Descriptions in General 

Ani makes films in response to film descriptions. These descriptions are very 
high-level, typically vague and incomplete, and describe the story or action, the 
characters, and the kind of film desired. A user (or if you prefer a collaborator) of 
Ani presents the description in a key-word nested notation. Ani would be a more 
impressive if the system could read a story and then animate it. It is probably 
within the state of the art to have a film description automatically generated by a 
story understanding system ([Schank 1975] for example). Stories of the complexity 



I. Of course if Ani were extended as described in Subsection I "Adding Appearances to Ani" on page 
16? then the sizes and shapes of the characters would be an important aspect of the films. 
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of Cinderella have been understood by story understanding systems (e.g. [Rieger 
1977]) well enough so that they probably could be extended to produce film 
descriptions, though such systems are currently very limited in generality. The 
scene descriptions are especially amenable to automatic generation from stories. A 
system reading that Cinderella's stepmother refuses to help Cinderella obtain 
clothes for the ball and orders her to stay home and not go to the ball could 
describe that as the stepmother preventing Cinderella from going to where the 
Prince is. it is more difficult to automatically infer the character descriptions, 
however many stories explicitly provide them (e.g. "Her evil stepmother then ..."). 
A more complete version of Ani connected to a good story understander could be 
asked to make a film of a children's story or fairy tale and the story understander 
would read the story and provide Ani with a film description based on the story. 
Film descriptions could even be automatically generated along the lines of Talespin 
[Meehan 1976], 
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There are four parts to a film description. Abstractly they correspond to the 
description of the major parts (the characters in Ani's films), their relationships, 
their interactions (the events in the scenes), and global constraints. They are 

(a) The character descriptions which consist of a series of physical 
(e.g. ugly), personality (e.g. shy), motion (e.g. graceful), and "role in 
story" (e.g. heroine) descriptors for each of the film's characters. 

(b) Descriptions of the relationships of the characters in the film (e.g. 
hates, dominates). 

(c) The description of the scenes of the film. A scene consists 
primarily of a partially ordered series of events. There are three 
types of events that Ani can accept: events to establish either the 
personality, emotional state or relationship of some characters, 
events to convey the interaction of some characters, and events to 
display the behavior of some characters. 

(d) The remaining part of a film description is a list containing the 
desired levels of global film descriptors such as variety, complexity, 
subtlety, energy, and the length of the film. This last part also 
includes the focus which provides some priorities and guidance in 
the control of Ani's decision making. 
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Character Description Relationship Description 



| Personality | 

I Appearance | 

| Movement | 

| Role in Story j 



Scene Description 

| Behavior 

| Emotional State 

I Interactions 



Dominance 
Affection 
Acquaintance 



Global Description 
| Descriptor Levels 



Focus 
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Figure 3.1 Different Parts of the Film Description 

Section C Film Descriptions as Programs 

The standard view of Ani in this thesis is that Ani creates a film after a user 
declares what it should be like by providing a film description. Another view is 
that Ani is really a very high-level interpreter for a special purpose 
programming language. From this view the film descriptions are really programs. 
This is somewhat different from the view that Ani is really the programmer, that 
the film descriptions are the specifications, and that the fully elaborated film 
descriptions produced are really the programs. 
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The film description programming language is simple. To create a character 

the "programmer" just fills in the following template. 

( define <the characters name> character 
(process initial description 
( < descriptor- type> <description>) 

( ^descriptor- type> <description> ) ) ) 

The currently defined descriptor-types are physical, personality, movement and 
"role-in-story". The description that follows each type is either a single 
descriptor such as "ugly" or a conjunction of descriptors in a list such as "(and 
good rriendly hard-working shy)". The descriptors themselves are discussed in 
Section D "What the Words in the Film Description Mean" on page 81 and are 
definable as described in Appendix IV. "Descriptors as They Look in Ani" on 
page 264. 

The relationship of two characters is described in two parts — how the 

first character relates to the second and how the second relates to the first. 

The template for defining relationships is similar to the character definition. 1 

(define (relationship-of < first character <$econd character>) relationship 
(process initial description 
<de$cription>)) 

An example of a description is (and hates dominates). 



I. The relationships are themselves actors that can be accessed by anyone who knows of them (which 
includes at least the characters involved). 
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The scene descriptions are more complicated and ire defined as follows. 

( clef i no < scene name> scene 
(process initial description 
( < ac tion de scrip tion> ) 
(set your length to <short, medium or bng>))) j; medium is default if left out 

An "action description" is either an event specification, a sequence of event 
specifications denoted by a list of event specifications preceded by the atom 
"sequence:", or a set of event specifications (i.e. let Ani determine the order 
perhaps displaying the events concurrently) which is denoted by an event 
specification list preceded by "and". 

(a) A list beginning with "establish" followed by either the personality 
of a character (e.g. (personality stepmother)), a relationship (e.g. 

^^^^ (relationship-of Cinderella stepmother)), or an emotional state 

• description (e.g. (emotional-state Cinderella (joy (negative 

high)))). 

(b) A list beginning with "convey" followed by a triple of the action, 
the actor, and the other actor involved or another event 
specification (e.g. (convey (wants Cinderella (meets Cinderella 
prince)))). 

(c) An "event specification" can recursively be an action description 
(i.e. a sequence or set of event specifications). 
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The final part of a film description are the global constraints and the film's 
focus. Ani has default values for all the parameters involved and they are 
typically overridden. The global constraints are values (currently either "high", 
"medium", or "low") for the level of variety, complexity, originality, coherence, 
obviousness, flashiness, and energy. These are all represented as actors that 
are asked for suggestions. The values are also used by many of the 
choice-making components of Ani. For example, the part that decides how many 
times to repeat an action will tend towards more repetitions if the obviousness 
level is high and the variety level is low, The focus is an actor that contains 
the parts that are to be emphasized. In the Cinderella film description 
Cinderella is part of the focus, as is her relationship to her stepmother, and the 
second and third scenes. Also the focus gives priorities which order the way 
the dynamics of the characters is determined. The Cinderella film description, 
for example, specifies that the coherence of each of the characters {rather than 
their relationships or the scenes) is to be given priority. 

Section D What the Words in the Film Description Mean 

The words in the film description look like English and yet Ani doesn't 
understand English, so what exactly do these words mean to Ani? The meaning 
associated with each of the descriptors in the film descriptions is the minimum 
needed by Ani in making films. All that Ani knows about the word "evil" is that 
it suggests that an evil character move in short jerky motions, that its 
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movements be deliberate, its activity high, and to establish a character as evil 
one can try conveying that that character causes something that it considers a 
"bad thing", or that it hurts or chases someone else that it considers its enemy 
or a stranger. 1 Ani cannot break apart the concept of evil into more primitive 
components nor know what its like to be evil. It is possible that Ani's films will 
never be of high quality until Ani possesses a better and more complete 
understanding of such concepts. Some might argue that Ani could never produce 
true Art without feeling the feelings that portrayed (or expressed) in the films. 
I argue that one need not feel those emotions in order to behave as if one did 
in Subsubsection b. "Lack of Feelings" on page 229. 

Ani has a very limited understanding of the words used in these 

/"""^ descriptions. A few of the words in the film description are Director primitives 

such as "define" and "set your". Oirector, the language that Ani was 

implemented in, is described in [Kahn 1978b]. These words have procedural 
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I. Some people may be concerned that what Ani knows about evil is not correct, for to them an evil 
character moves slowly, more like a snake perhaps, and is both calm and neurotic. Ani's knowledge 
ba<e could be changed to match each viewer's idiosyncrasies, (if the changes require new kinds of 
descriptors, major modifications to Ani might be necessary.) What is interesting about suggestions is 
that they can be only partially right or only weakly correspond to what they are supposed to and the 
resulting' product is still good. This is because the suggestions interact with one another and because 
there are often many suggestions for the same purpose that tend to help and reinforce each other. 
That good characters might possibly behave the same as evil characters (e.g. also hurt their enemies) is 
a serious problem only to the extent that other aspects or behavior of the characters are unable to 
convey the difference. 
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meanings stating such things as whether what follows should be inserted in a 
database, extend the behavior of the actor in question or initialize some 
variable. Some of the others such as "process initial description" and "set your 
level of" are Ani's extensions to Director. These too are procedural and their 
meaning relates to the specific representations of such constructs as suggestions, 
character and relationship descriptions, and global constraints. The remaining 
words in the description are the interesting ones. They are descriptors and are 
all represented by computational actors in Director. This is appropriate since 
the descriptors need to have their own memory and be able to respond to a 
variety of messages. 

The descriptor actors are called upon to provide suggestions for values and 
methods, to help in the making of comparisons, and to check prerequisites and 
optimality conditions for methods. They are also the repository for statistics on 
their occurrence which is used to determine the uniqueness of the entities that 
they describe. The descriptors are linked to each other by "opposite" and 
"similar" links and these are used for both comparisons and indirect sources of 
suggestions. 

A few of the descriptors are currently little more than reminders that they 
should be taken into account. For example, Ani carefully determines how 
"deliberate" a character's movements should be and then never takes that into 
account in later decisions. It should at least influence things such as how often, 
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how quickly and to what extent the character in question changes direction. A 
discussion of why Ani was built this way is in Subsection 3 "Attending to Myriad 
Details" on page 184. 

A paraphrasing of the definitions of a few descriptors in Ani follow. 1 



"Shy" is a personality descriptor of characters. It weakly suggests a low 
speed. It strongly suggests an avoidance of enemies. It strongly suggests a 
great avoidance of strangers. A slight attraction to friends is weakly suggested. 
It suggests that one method for establishing that a character is shy is to convey 
that the character avoid some other character that is a stranger to him or her. 
Shy is considered to be the opposite of "brazen" and "forward". 

"Hates" is defined as a (negative) affection descriptor of relationships. 
Hates suggests that one of the following methods be employed to establish that 
one character hates another. 



I. To find out what this really looks like inside of Ani read Appendix IV. "Descriptors as They Look 
in Am" on page 264. 
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(a) Convey that the character avoids the othei. This is moderately 
strongly suggested and is best when the character does not 
dominate the other. 

(b) Convey that the character hurts the other. This is best when the 
character dominates the other and is strongly suggested. 

(c) Convey that the character pushes the other. This too is best 
when the character dominates the other but is only weakly 
suggested. 

Hates is the opposite of "loves" and is similar to "despises". 

The "variety level" is a global descriptor of a film (or scene). Currently it 
can have three possible values, "high", "low" and "medium". It weakly suggests 
that if the variety level is high pick the least used value for that element, if it 
is low pick the most used value, and if medium pick the median as a value. For 
example, if the variety level is high and a suggestion for the speed of a 
character is being requested and the speed of three of the other characters has 
already been decided to be fast and medium then the variety level will suggest 
a low speed. 

The word "meets" is a special case of "moves" which is a kind of activity. 
Among the prerequisites for X going to meet Y is that Y be on stage. X need 
not be on stage since it can come on and then meet Y. Others prerequisites 
are whatever prerequisites there are for X moving to the vicinity of Y (which 
"Moves" knows about). Meets knows that the first part of a meeting is X 
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moving a small part of the way towards Y. Meets strongly suggests that to 
convey that X meets Y that one display X moving to the vicinity of Y. The last 
suggestion could be improved by being changed to a sequence of one moving 
towards the other followed by some interaction of the two characters. 

It is interesting to note the extent to which Ani's definitions of descriptors 
are procedural. They "mean" what Ani knows about how to convey them and 
how they can be used procedurally as a source of suggestions. The definition 
of a descriptor contains a little information as to how it relates to other 
descriptors and otherwise lacks any declarative aspects. Associating suggestions 
with the descriptors and representing the descriptors with actors lead to a very 
modular system. This organization chunks the knowledge so that no large 
searches through a database are necessary. This is important were Ani ever 
enlarged to know much more. 

Section E What Ani Turns the Film Description into 

Ani takes the Cinderella film description and turns it into a very detailed 
description capable of being run by Director, [Kahn 1978b] an unintelligent 
graphics system that makes no real choices. This detailed description of the 
Cinderella film is best thought of as a computer program. The program is 
written by Ani and executed by Director. It need never be seen by human 
eyes and therefore little attempt was made to make the program very legible. 
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To give an idea of the level of detail needed to run the film the 
paraphrasing of a short portion of the second scene generated by Ani follows. 1 
The first subscene of the "kept-apart" scene (where it conveys that Cinderella 
wants to meet the Prince) is defined. It is estimated to take a little more than 
a second. Cinderella's dynamics in this subscene should be as follows. She 
should move in a moderately jagged path with little curvature and each curve 
should be moderately long. She should move very deliberately and somewhat 
repetitively and quickly get up to a high speed. She should in general be very 
active and move in a very rhythmic manner. Cinderella should initially be placed 
in the upper far right of the screen while the Prince should be at the opposite 
corner (the lower far left). Without any delay Cinderella should begin to turn to 
face towards the Prince. After facing him, she should move a small part of her 
distance to the Prince. If he moves she should keep moving towards his new 
location as she walks (he does not happen to move in this part of the scene). 

A few of the. details require explanation. Most of the values are still 
symbolic and yet to draw on the screen Director needs numerical values for 
parameters such as the speed, location, and path length. These symbols are 
translated into numbers by an interface that is part of both Ani and Director. 
The translation of expressions such as "upper far right", "small way from the 



1. The actual code produced can be found in Appendix VI. "Some Code Written by Am" on pa^e 

272. 
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vicinity of Cinderella in the direction of the Prince" and "high acceleration" into 
numerical values in done in manner that is sensitive to the dimensions of the 
display and parameters for the normal amount of time to cross the display and 
to turn around completely. This is described in greater detail in Appendix II. 
"The Running of the Cinderella Film" on page 250. 



<-s 



Page 88 






i 



/**N 



/""\ 



«#*"**% 



Suggestions 

Chapter IV Suggestions 

This chapter describes suggestions, the most important form of knowledge dealt 
with by Ani. Suggestions are constructed by descriptions of the characters, their 
relationships, comparisons of them, and global aspects of the fiim or scene. Ani 
gathers suggestions when faced with a difficult choice. Typically Ani finds many 
suggestions and spends much time trying to make sense out of them to determine 
which ones to follow. Ani treats differently the suggestions for values of elements 
of a detailed description of the characters and the suggestions for the activities or 
events that should occur within a scene. 

Section A What Suggestions are 

A large portion of Ani's time is spent gathering and combining suggestions. 
Suggestions from many different sources are gathered in order to choose events 
and activities and to flesh out the descriptions of the characters. A suggestion for 
a particular element (such as Cinderella's typical speed) consists of a suggested 
value (e.g. "low") and the strength with which it is suggested. The source of the 
suggestion is also connected to each suggestion. A suggestion for a method for 
establishing or conveying something has as its value a complex description of an 
activity or event. For example, a suggestion for how to convey that a character 
prevents another from moving is that the character perform the activity called 
"standing guard" over the other. 
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What Suggestions are Suggestions 

Suggestions are treated by Ani as entities to be gathered, considered, 
elaborated, combined, rejected, and compromised. Among the many sources of 
suggestions for the elements of a detailed character description are the character 
descriptors, the relationship descriptors, the character comparisons, and the film 
descriptors. The sources of suggestions for methods or activities are abstractions 
of tasks to establish, convey or display some event, emotional state, personality, or 
interpersonal relation. This is described in greater detail in the next two chapters. 

Section B The Representation of Suggestions 



A suggestion typically consists of four parts. 



/-\ 



(a) The first part describes the subject of the suggestion. Examples 
are "Cinderella's typical acceleration", "the relative speed of 
Cinderella and her stepmother", and "conveying that the stepmother 
stops Cinderella from being able to move to the vicinity of the 
Prince". 

(b) Next is the value suggested such as "high", "greater than", or a 
description of the activity of someone "standing guard" over 
another. 

(c) The third part indicates the strength with which the suggestion is 
being made. The strength is a crude estimate of the reliability of 
the suggestion. This is just one of several factors taken into 
account when deliberating conflicts between suggestions. 
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(d) The final part is the source of the suggestion. This is used by Ani 
in weighing the importance of a suggestion. To a limited extent 
Ani will not count more than once different suggestions that are 
from equivalent or related sources. The suggestion source is also 
used in the justification of Ani's choices. 

As suggestions are combined the source becomes a list of the sources and the 
strength the average strength of each suggestion. A suggestion can be 
extended as needed to have more parts. For example, if a suggestion is 
rejected, then the reason for the rejection is added. 

The inclusion of the stereotypicality of a suggestion would be very useful 
for making choices based upon the desired level of originality and clarity. 1 
Stereotypical values would be preferred if the desired level of clarity is high or 
the level of originality is low and avoided otherwise. 

Section C Value Suggestion Sources 

One of the major problems that Ani faces is what value to choose for an 
element such as the typical speed of a character. Values are chosen by 
gathering up suggestions from various sources, combining those suggestions that 
are similar enough, noting and classifying any conflicts. If the suggestions found 
do not conflict, Ani has it easy and just follows the suggestions. If there are no 



I. This one of many improvements that should be made to Ani. 
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suggestions Ani looks for suggestions for the value of the element relative to 
the value of that element of another character. For instance, the stepmothers 
dominance of Cinderella suggests that she be faster than Cinderella (i.e. have a 
greater typical speed). When a conflict arises between suggestions Ani has 
several ways of resolving it as is discussed in Section C "How Conflicts are 
Resolved" on page 108. 

Suggestions for the value of an element of a character's dynamics (or 
appearance in an extended Ani) come from the descriptions of the character, the 
characters relationships and comparisons with the other characters, and the 
global description of the film. The order in which they are gathered and their 
relative priorities are strongly influenced by the film's focus. The role of the 
i^**^ focus is described in Section D "Postponement and the Focus" on page 110. 

The descriptors that define a character's personality, physical appearance, 
movement and role in the story often make suggestions for values. For 
example, the descriptor "hard-working" applied to a character suggests the 
typical speed of that character not be slow, that the level of activity of the 
character and the deliberateness and repetiveness of the character's movements 
be high, and that the degree of rhythm in the character's movements not be low. 
Lazy is considered an opposite of hard working while industrious is considered 



f* m % i \ 



Page 92 

! 



/"N 



/*\ 



/"N 



: * 

Suggestions ~ Value Suggestion Sources 

similar and these provide indirect suggestions. 1 

The descriptors of the relationship of the characters and the comparisons 
between the characters are the sources of suggestions for the relative values 
of an element. For example, the relationship descriptor "dominates" suggests 
that since the stepmother dominates Cinderella that the stepmother's typical 
speed should be greater than Cinderella's. These relative value suggestions 
produce absolute value suggestions when the value of either of the two 
characters involved is determined. So if Ani decides that Cinderella is to move 
slowly then the previous relative value suggestion in turn suggests that the 
stepmother's speed be greater than low. 

Ani compares the descriptions of characters to determine how similar or 
dissimilar are the personalities, physical appearances, and movements of each 
pair of characters. 2 Ani compares two characters by inspecting the physical (e.g. 
ugly), personality (e.g. shy), movement (e.g. graceful) and "role in story" 
descriptors of each. This part of Ani finds out how often the characters have 
similar descriptors and how often they have opposite descriptors. The use of 



1. Of course the values suggested by "lazy" must be inverted to be appropriate for a character 
described as "hard working". 

2. If there were many characters and especially if many had nothing to do with the others then 
making every pair wise comparison would be wasteful. Ani could easily be changed to make 
comparisons only between characters that are related (i.e. their relationship has been explicitly 
described). 
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the descriptors is recorded to determine how frequently each descriptor is used. 
The ways in which characters relate to each other are also compared. For 
example, if Character A hates Character B while Character B likes Character A 
then the difference is noted as evidence for the dissimilarity of the two 
characters. To summarize the findings Ani groups the statistics according to 
their type (e.g. personality, physical, or all of them combined). Findings relating 
to little used descriptors are considered more significant in making these 
summaries since if a particular descriptor applies to nearly all characters, then it 
indicates little about the relative similarity of two of them. While making these 
comparison summaries Ani also notes any character or aspect of a character that 
is unique. The stepmother's personality, for example, is unique since the other 
three characters are good and kind. 

These comparisons and uniqueness studies provide many relative 
suggestions. For example, if two characters are very similar, then that strongly 
suggests that they have the same values for most elements. Similarly the 
finding that a character is unique suggests that that character move (or appear) 
differently from the others. This is too simple-minded, of course. Sometimes 
what is crucial is to portray subtle differences between similar characters. 
Other times the difference between two similar characters should be accented 
by choosing divergent values for one or two elements to help the viewer 
distinguish the two characters. Currently Ani is too limited to choose which 
aspects should be used to convey the similarities or differences between two 



j#*\ 



Page 94 



i \s 



j -a 

Suggestions ~ Value Suggestion Sources 

characters, but instead tries to use all the aspects. 1 

Descriptors that apply to the film as a whole are also a good source of 
suggestions. For example, when the energy level is high it suggests high values 
for elements of a character's movement such as speed and acceleration. When 
the desired variety level is high it suggests values different from those chosen • 
already. 

If in the rare event that no suggestions are found then a default value is 
obtained from the element itself (e.g. size or speed). 

Section D Combining Value Suggestions 

■n*v After gathering up some of the suggestions for the value of a descriptor Ani 

combines them, noting any conflicts between the suggestions. The combination of 
these suggestions proceeds as follows, 



1. The heuristics to do this should be added if Ani is extended to determine more aspects of the 
characters of the films (e.g. their size, shape, color and so on). 
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(a) Combine those suggestions that are identical or nearly so into one 
suggestion with a multiple source and an averaged strength. 

(b) Combine those suggestions that can be combined without any loss 
of information. E.g. the values "less-than high" and "medium" 
combine to "medium". 

(c) Note and classify any conflicts between incompatible values. 

(d) Compromise any suggestions whose values are "close enough", e.g. 
"medium" and "low" even if information is lost. 

(e) Resolve any remaining conflicting suggestions as described in 
Section C "How Conflicts are Resolved" on page 108. 



There is an expert for each set of possible values a descriptor may have 
that answers questions about whether two values are compatible, incompatible, 
or compromisable (somewhere in between the first two). These experts also 
are responsible for compromising two values and classifying the type of 
incompatibility if one exists. The simplest such expert knows just three values 
that correspond to "low", "medium", and "high" (though they may actually be 
"short", "medium" and "long"). The most complex one in Ani accepts these three 
values optionally modified by "less-than", "greater-than" and "different-from". 
Another expert handles relative values of the form "less than so-and-so" and 
"different from so-and-so". 
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To actually choose a value Ani analyzes the results from the combination 
step. If no conflicts are discovered and if there are enough strong suggestions, 
then the value suggested is accepted. 1 If there are not enough suggestions, 
then Ani gathers up more information, if possible, and the process recurses. If 
no more suggestions can be found because other choices have yet to be made, 
then those choices are given higher priority, the current choice is postponed, and 
the reason for its postponement is recorded. (Postponement of choices is 
described more fully in Section D "Postponement and the Focus" on page 110.) 
If conflicts were found then a series of heuristics are tried to resolve the 
conflict by either forcing a compromise or by rejecting some of the conflicting 
suggestions. 

Eventually a value is chosen. The decision as to which choice to work on 
next is based upon the idea that those choices that are the easiest to make, i.e. ' 
those that have the most non-conflicting suggestions should be made first, 
followed by those that are lacking essential (but obtainable) information, followed 
by those with conflicts, and those which have the least basis should be made 
last. When more than one choice is appropriate to work on then the focus will 
order them if possible. The ordering of these choices is discussed in Section D 
"Postponement and the Focus" on page 110. 



I. "Enough" in this context means if the measure based upon the number of suggestions, their sources, 
and their strengths is at least as good as the minimally good suggestion provided by the film's focus. 
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Section E Method Suggestion Sources 

Of importance equal to that of the value suggestions are the method suggestions. 
They indicate what activity should occur or which method should be applied to 
convey the personalities, relationships, or interactions of the characters. A few 
method suggestions are usually made by the task of the section of the scene 
being elaborated and some of those suggestions in turn often suggest other 
methods. For example, in the second scene there is a subscene whose task is 
conveying that Cinderella is kept from meeting the Prince. To Ani this is 
expressed as "(convey (prevents step-mother (meets Cinderella prince)))". 
"Prevents" is an actor who is asked for suggestions as to how do this. It 
knows only one way to have a character prevent some activity, namely to have 
that character undo at least one of the prerequisites of that activity. And so it 
asks "meets" for the prerequisites of Cinderella meeting the Prince. "Meets" 
replies that the Prince must be "on-stage" and that "moves" should be asked for 
the prerequisites of Cinderella moving to the vicinity of the Prince. "Moves" 
replies that the only prerequisite it knows about is that Cinderella "can-move" 
to the place. At this point the method suggestions are 



Page 98 



/****^. 



/-\ 



J^^^y 



^*\. 



Suggestions Method Suggestion Sources 

(a) Do something to convey that the stepmother causes Cinderella's 
inability to move to the vicinity of the Prince. 

(b) Do something to convey that the stepmother causes the Prince to 
no longer be on stage. 



Ani then goes through each suggestion and gathers up suggestions for 
accomplishing them. In this case "undoes" is the only one to ask for suggestions 
and it suggests that the stepmother chase the Prince off the screen in order to 
undo the fact that he is "on-stage". To convey that *he stepmother causes 
Cinderella's inability to move to the vicinity of the Prince, it suggests that the 
stepmother make a "fence" between Cinderella and the vicinity of the Prince, 
that she stand guard between Cinderella and the Prince or that she kill 
Cinderella. The only suggestion that needs further suggestions as to how to be 
accomplished is that the stepmother kill Cinderella. Ani has no idea how to 
convey this (because the "kills" actor is very incomplete) and so the suggestion 
is eliminated. 1 How Ani chooses among the remaining suggestions is described in 
Section B "How a Method is Chosen" on page 119. 



1. A m's use of knowledge is robust. If parts are missing Ani manages to go on rather than stop due 
to ignorance (not knowing how to convey a murder, in this case). This is analogous to humans who 
think of a valid solution but know not how to pull it off. If the "kill" actor had been completed and it 
could run in the allotted time then it might be chosen. The selection of the stepmother killing 
Cinderella fits very well the relationships involved, and since the fairy godmother is described as 
magical she could "resurrect" Cinderella in the next scene. 
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Ani distinguishes those methods that can just be displayed, such as the 
"stands-guard", "chases-off", and "makes-fence" activities, and those that need 
further thought (i.e. more suggestions need to be gathered to find out how to 
carry thern out). The former are called "display" methods while the latter are 
"convey" methods. The convey methods are treated the same as the convey 
tasks given in the original film description. 

A typical source of suggestions for what to convey are personality 
descriptors such as "good". "Good" strongly suggests that to establish that a 
character is good one should convey that the character helps some other 
character. That other character shouid ideally be described as weak and good 
{and not be the character in question of course). "Good" also suggests that one 
can convey that the character prevents some event from occurring when that 
character considers the event to be evil. 

While instantiating the suggestions from such sources Ani must choose the 
events or characters described. If Ani fails to find any character that meets the 
specifications of a suggestion then the suggestion is ignored. 1 An extension of 
Ani that is interesting to consider is how Ani could create a new character when 



I. In the previous example this was not a problem since the only specification is that the character to 
be helped should not be the one who we are trying; to establish as good. The method specifies that 
ideally a weak and good character should be chosen, but it will be satisfied with a character that is 
neither weak not good, if need be. 
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no character exists that meets the specifications. This is similar to what 
happened in the Disney version of Cinderella. Nowhere in the original story is 
there mention of the mice that Cinderella helps. One of the reasons Disney 
created them was to help establish Cinderella's personality. In achieving some 
high level goal (showing that Cinderella is good) Ani could be extended to 
introduce a new character who is weak and good and helped by Cinderella. 

Section F Relating Method Suggestions 

Method suggestions are not combined as value suggestions are. The main reason 
is that they are used to decide what to do rather than determine a value for 
some descriptor. Ani must choose a value and only one, but may choose any 
/"""n number of methods to accomplish something or can even skip that part and not 

select any methods. The step that is analogous to the combining step for value 
suggestions is the relating step of method suggestions. The pool of suggestions 
gathered for a subscene are grouped according to whether they are identical, 
similar, opposites, specializations, or generalizations. Within one subscene many 
different things may' be happening, for example, one character may be meeting 
another while the other is having its personality established. Clearly Ani does 
not want to plan to do contradictory activities and eliminating such suggestions 
rules out a few. If a method is suggested more than once or is a specialization 
of, or is similar to, another then it is preferred. This reflects an aesthetic bias 
towards the economy of accomplishing several tasks with one method. Chapter 
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VI "Choosing Methods" on page 116 describes the w>iy in which methods are 
chosen in more detail. 
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Chapter V Choosing Values 

This chapter describes how Ani chooses values in the process of creating a detailed 
description of the default dynamics of the characters in a film. Dynamics 
descriptions are broken into elements and the choice of a value for the elements is 
pursued in quasi-parallel. The hard problems that Ani faces here are the 
coordination of the interdependent simultaneous choices and the resolution of 
conflicting suggestions. The mechanisms described here would probably be used if 
Ani were extended to determine the appearances of the characters. 

Section A What is a Value? 

Rapid movement gives the impression of "violence" as opposed to the 
"gentleness" of slow motion. A sudden slackening of speed or a 
momentary pause in movement gives it a mark of "hesitation". Sudden 
and repeated variations of direction, or even merely of speed, give the 
impression of "nervousness" or "agitation", etc. 

-Albert Michotte [Michotte 1950] p. 118 

Does he [a typical Disney animator] want a fast sneak or a slow sneak? 
A fast one might put over the idea of cunning better. 

Bob Thomas [Thomas 1958] page 139 
To portray an animated character Ani is faced with many choices. Should the 
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character move slowly or swiftly? Should it move very deliberately or uncertainly 9 
Should it be faster than some other character? Should it tend to avoid others? 
Should it move very directly or in graceful curves? Should it appear large or small, 
rounded or pointed? All these questions can be answered by choosing among a few 
permissible values for the element in question. Currently Ani knows a number of 
elements for the typical or default dynamics of a character. The Ani's mechanisms 
for choosing values for these elements should work well for other elements such as 
the size, shape, color, and texture of the characters. There are no theoretical 
reasons why Ani cannot handle these other elements. 1 

The process of choosing a value for an element differs greatly from the 
choosing of events and activities (which is described in the next chapter). An 
element has only one value which is selected from a small set of possible values. 
This contrasts with the choosing of activities where there are very many different 
possible activities and Ani sometimes chooses more than one for the same task. 
Another major difference is that there usually exists relative values for elements 
while nothing corresponding exists for methods or activities. For instance, Ani 
decided that the typical speed of Cinderella should be slower relative to that of her 
stepmother. 



t. Because one should only devote so much effort for demonstration purposes in the implementation 
of a prototype, 1 decided to concentrate on the essence of animation, namely motion, and leave the other 
aspects of animation underdeveloped. 
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This breaking of the dynamics (or appearance) of a character into elemental 
parts builds certain prejudices into Ani. Something as complex as the dynamics of a 
character can be broken up in many different ways, Ani divides the dynamics into 
speed (and acceleration), path (length, curvature, smoothness), style (e.g. 
repetiveness, deliberateness, level of activity) and tendencies (e.g. to avoid 
strangers, to be attracted to strangers). A different way of breaking up the 
dynamics would result in a different style of animation. 1 

Ani creates a typical dynamics of a character to give it some character or 
identity. The personality, mood, desires and activities of a character are all 
conveyed by the motion of that character in the pure animation that Ani makes. 2 
The typical dynamics interact in a complex manner with display methods to produce 
the actual motions of a character. A simple example is when a character typically 
moves slowly but during a chase scene moves much faster than usual. The way this 
happens is described in detail in Section C "How a Method is Elaborated" on page 
122. 



1. There are of course many other factors that contribute to the style of an animator as is discussed in 
Section C "Ani and Style" on page 239. 

2. Of course in more conventional animation one can also use the appearance, facial expression, 
dialog, or color to do this. 
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Section B How a Value is Chosen 

The process of filling out a detailed description of the typical dynamics of a 
character consists mainly of repeatedly choosing a value for a different element of 
the dynamics from a small set of permissible values. Ani goes about this by 
gathering up suggestions (as described in Chapter IV "Suggestions" on page 89), 
combining them, resolving any conflicts, and finally choosing a value. 

To illustrate this, a trace of the process of choosing of a typical speed for 
the stepmother is presented in this section. Speed is an especially simple element 
because it has only three values, "low", "medium", and "high". 

Ani begins by creating a choice point which is an actor that keeps findings, 
partial results, a history of work on this choice, and a justification of the final choice 
for the stepmother's speed. Suggestions are then gathered from the description of 
the stepmother and the suggestion of a high speed from "strong" is the only one 
found. Ani does not consider this a sufficiently good reason to choose a high speed. 
The focus of the film includes a sample of a minimally strong suggestion and this is 
the standard used to decide if a particular group of suggestions constitutes a 
"sufficiently good reason" to choose a particular value. Ani proceeds to gather up 
suggestions from the relationships and comparisons of the stepmother and the other 
characters. Choice points for the relative speed of the stepmother and Cinderella 
and for the stepmother and the fairy godmother are created and are asked for 



Page 106 



gtm* 



y"*"\ 



/*"*% 



/*"*N 



Choosing V; ilues How a Value is Chosen 

suggestions. They are found to be in agreement w.th a high speed for the 
stepmother. The first one suggests that the stepmother be faster than Cinderella 
which is compatible since Cinderella is slow. The decision that the stepmother be 
faster than Cinderella is nearly as complex as this choice of the stepmother's speed 
and is based upon their relationship and a comparison of the two characters. The 
choice point for the relative speeds of the stepmother and the fairy godmother 
suggests that the speeds of the two be different (since Ani recognizes that the 
characters are so different). Since the fairy godmother's speed is medium this 
suggestion is compatible with a high speed for the stepmother. 

There are suggestions other than these three suggestions, however, which 
conflict with the idea that her speed be fast. For example, the comparison of the 
stepmother and the Prince suggested that their values be different and yet his 
speed is also high. This conflict eventually gets resolved and the suggestion for the 
relative speed of the stepmother and the Prince is ignored because it is much less 
strongly suggested. 1 



I. This is due to a very large difference in the strength of the suggestions. If this had not been the 
case Am would have more work to do to resolve this conflict. See the following section for a discussion 
of how conflicts are resolved. 
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Ani is still not very happy with this state of affairs and gets suggestions from 
the film's global descriptors. More conflicts are discovered. A compromise 
between suggestions of a high speed from the variety level and energy level and a 
suggestion for a low speed from the desired amount of flashiness is worked out. 
This is then combined with the previous suggestions and Ani chooses a high speed 
for the stepmother. 

Section C How Conflicts are Resolved 

Upon finding no conflicting suggestions for a value Ani just picks the value 
suggested. If in the rare event that no suggestions could be found, Ani just uses 
the default value associated with each element. More typically there are many 
suggestions and they don't all agree. It is important that she resolve these conflicts 
as sensibly as possible if there is to be much coherence to a character. 

Ani proceeds to resolve conflicts by taking each pair of conflicting 
suggestions and applying a succession of heuristics upon the conflict. If either of 
the suggestions was rejected because of their role in an already resolved conflict, 
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then it is rejected again. 1 Failing this Ani considers the relative strength of the two 
suggestions. Recall that associated with every suggestion is the strength given to it 
by its source and that as suggestions are combined their strength accumulates. If 
the strengths differ very much, Ani picks the stronger one. 

Ani next finds out the relationship of each of the conflicting suggestions with 
the most favored suggestions so far (only if neither of the two conflicting ones are 
among the favored suggestions). If one is compatible and the other not, then the 
compatible one is joined with the favored ones. Similarly if neither are compatible 
but one can be easily compromised with the favored suggestions it is compromised 
with the favored suggestions. The last part of this heuristic checks whether the 
f*\ favored suggestions are much more strongly suggested than either of the conflicting 

suggestions and if so rejects both of them. 

Finally, Ani tries to force a compromise between the two conflicting 
suggestions and failing that just picks the more strongly suggested one. The 
strength of the suggestions are a function of the number of sources, the original 
strengths of the suggestions, and the relative importance of the different suggestion 
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I. If this fails. Ani calls some never finished procedures. If they had been completed they would have 
tried to resolve the conflict by using criticism (either from the user or self generated). Failing that and 
if this choice was made before (as when Ani is being called upon to make variations of the same film) 
then Ani will either favor or disfavor the previously chosen values depending upon the desired degree 
of originality. 
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sources as indicated by the film's focus. 

Section D Postponement and the Focus 

The account of the selection of the stepmother's speed is accurate from the point 
of view of the choice point in question. From a more global point of view the 
choice point is often considered, postponed for other choices to be made, and then 
resumed again. Ani asks the choice points for the dynamics elements of each 
character to come up with values concurrently. As they run into trouble, especially 
if the trouble is the lack of values of other choice points, then a choice point is 
postponed. This scheme minimizes the arbitrariness of any choices that might result 
from the order in which the choices were made. It introduces two new problems to 
Ani though: when to postpone the efforts of a choice point and when to resume 
them. The philosophy behind this part of Ani is to order tasks so that the ones that 
are likely to be the easiest and most straight-forward run first, followed by those 
which are difficult but will not be helped much by the resolution of other choice 
points, and only when there are no other tasks to work on are the difficult ones 
that are lacking information run. This last situation is common and the focus is used 
to determine their ordering. 
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The selection of the stepmother's speed is an extreme example of this 
process since it was postponed ten times before finishing. When first asked to 
choose a value the choice point for the stepmother's speed found the suggestion 
from "strong" but nothing else and so postponed itself saying it was "not happy 
enough". Next time it was awakened, it iooked for relative values and soon gave 
up because the choice points for the speeds of the Prince, Cinderella and the fairy 
godmother were not making any better progress. 

Potentially, this situation could lead to a deadlock since the choice point for 
the stepmother is waiting for values from others who are in turn waiting for the 
stepmother's speed. Since a choice point explicitly represents the history of its 
reasons for postponement this is not a very serious problem. A choice point is not 
allowed to postpone if its postponement reasons are identical to the reasons for its 
previous postponement. 1 The only exception is if the previous postponement 
reason was that the values of some other choice points are missing and the most 
recent postponement reason of at least one of those choice points indicated that it 
was making some progress. This exception was added so a choice point could 
repeatedly be postponed because it is waiting for some other choice points which 
have not yet concluded so long as at least one of them is making progress. 



1. Actually choice points don't postpone themselves without asking a postponement manager for 
permission first. It is this manager that prevents this deadlock. 
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The third time that the choice point of the stepmother's speed was 
awakened, it found that it stiil needed the values of the other three characters 
speed but that the choice point for Cinderella's speed had made a little progress 
(that one went ahead without waiting for the values of the others since Cinderella 
is part of the film's focus). The stepmother's choice point for her speed was 
postponed three more times before the speed of Cinderella was chosen. It took 
three more awakenings before the other two characters had a speed. At that point 
the relative suggestions were added to the choice point. Conflicts were found 
among them and the choice point was postponed. Next time it woke up, it gathered 
up the global film descriptor's suggestions and tried to resolve conflicts but 
discovered that in compromising the global suggestions a new conflict was 
generated and so was postponed one more time. 

In deciding which choice point to awaken next Ani organizes the 
yet-to-be-decided choice points in a tree. The tree is organized according to 
advice from the film's focus. For example, it can be organized by character and 
then each character is further organized by type of element (e.g. motion descriptors 
and movement tendencies) or the type of element could be more primary and it is 
subdivided by characters. This choice is left up to the focus since a film can focus 
on different aspects, e.g. a film can give more priority to the coherence of the 
characters or be more concerned with, say, the (aesthetic) balance of the positions 
of the characters. 
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This tree of choice points is used by Ani to pick a subset of the undecided 
choice points to work on. Ani starts at the top of the tree and chooses between 
the immediate descendants by following the rankings in the film's focus and giving 
preference to those nodes which have the most choice points under them whose 
values are needed by others. The node selected is either a choice point, in which 
case it is awakened, or else is a node. If it is a node the process recurses and 
selects between the immediate descendants again. 

The focus is consulted here since the nodes that are worked on first are 
more likely to be coherent and locally optimal since they will not be constrained by 
the subsequent determination of the values of other choice points. Consider the 
situation in the previous example where one of the four character's speed had to 
be determined first to avoid a deadlock. Cinderella went first because the film 
focuses on her. As a result she received a speed that is consistent with her 
description and the film's global description and was not influenced by the speeds of 
the other characters. The other ordering rule prefers nodes that are holding up the 
largest number of other choice points. This tends to make Ani work first on the 
tasks that are the greatest bottlenecks to progress. 

This part of Ani is organized to maximize the amount of parallelism possible 
and, when not possible, to carefully order the choices. Each choice point can be 
created and explored almost independently of the others. Only when one needs the 
results of another that has yet to finish is there a question of postponing, resuming, 
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and deciding who should go first in cases of mutual dependencies. The large 
number of elements of a character's default dynamics multiplied by the potentially 
large number of characters in a film is the number of choice points that could be 
explored in parallel. 1 It is interesting to contemplate if human animators conceive 
their films sequentially or if they exploit this potential for parallelism. Section A 
"The Simulation of Ani's Characters" on page 251 and Section B "Running Activities 
in Parallel" on page 253 discuss in detail other roles parallelism has in Ani. 



I. The number would increase further if other types of elements of a character's definition (e.g. the 
elements of character's appearance) were considered. 
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This chapter describes Ani's algorithm for choosing between methods for animating 
the activities, interactions, and personalities of the characters. The problems of 
temporally ordering those methods chosen is discussed. Also of importance is the 
process of elaborating the methods once chosen to produce executable descriptions 
of the activities that are to occur within a scene. 

Section A What is a Method 

In addition to choosing values of elements Ani must choose among methods to 
accomplish tasks. Ani recognizes three levels of tasks. 

(a) Tasks to establish the personality, relationships and emotional state 
of the characters. 

(b) Tasks to convey the interactions and high-level behavior of the 
characters. 

(c) Tasks to display the actions of the characters. 

A suggestion to accomplish some task can only suggest a method at the 
same level or one lower. Typically the suggestions for how to establish 
something suggest in turn what should be conveyed. The majority of the 
suggestions for how to convey something are to convey something else (or a 
sequence of other things). The remainder suggest some activity or event to be 
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displayed. Display methods need no suggestions to carry them out since they 
correspond to the primitive activities that Ani can animate. They do, however, 
need to be tailored to the particular situation by specification of the locations 
and precise dynamics of the participants of the action and the number of 
repetitions of any subactions. 

Examples of these different levels should help clarify this. One of the 
suggestions to establish that, say, the stepmother hates Cinderella is that one 
convey that the stepmother hurt Cinderella. To convey that she hurts Cinderella 
there is a suggestion that she display the stepmother coming in contact with 
Cinderella and repeatedly hitting her. This activity is further elaborated by 
determining the dynamics of the stepmother and Cinderella and the number of 
times the stepmother will hit Cinderella. The details of this are presented in 
Section C "How a Method is Elaborated" on page 122. 

Method suggestions include not only a description of what to display or 
convey but also their prerequisites. They often also describe conditions that 
should be true for this activity to be most appropriate. For example, the 
previous suggestion that the stepmother hurt Cinderella in order to establish 
that she hates her is best when the stepmother dominates Cinderella (which 
happens to be the case). The prerequisites are important for eliminating 
inappropriate methods and the optimality conditions are used to help select 
among a large pool of possible methods. 
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The process of obtaining suggestions of a successively lower level of 
abstraction reverses the process of abstraction that went into the making of the 
film description from the Cinderella fairy tale. When I wrote the film description 
i was constantly taking specific events in the fairy tale and translating them into 
very high-level intentions. For instance, the description of the scene in which 
the stepmother does something which prevents Cinderella from meeting the 
Prince was abstracted from the episode in the fairy tale where Cinderella is 
forced to stay home and not allowed to go to the bail. 

Ani takes a very abstract description and constructs a very detailed one 
that often has little resemblance to the original story. This is one of the the 
main reasons why the Cinderella film deviates so much from the original fairy 
tale. If a film is a loose interpretation of a book then probably a very similar 
process took place in the making of the film. The filmmakers read the book, 
created an abstract high-level description, and then were forced by the realities 
of film making to provide a highly-detailed product. The degree of looseness of 
an interpretation of a book may depend critically on how much abstraction 
occurred before the subsequent elaboration. 
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Figure 6.1 Elaboration and Abstraction of Film Descriptions 

Section B How a Method is Chosen 

After gathering suggestions for what should happen during a subscene Ani must 
select a small number of them to actually follow. In addition, if more than one 
is picked then Ani must decide whether the events they describe should happen 
simultaneously or sequentially (and if sequentially in what order). 
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Ani begins with the suggestions for the establishing things. Typically there 
are more suggestions than is reasonable to follow and so suggestions must be 
eliminated until a reasonable number is left. First Ani tries to make sense out 
of the suggestions gathered by discovering how they relate to each other. 
Suggestions that are identical, very similar, opposites, or are specializations or 
generalizations of each other are grouped together. Ani eliminates those 
suggestions whose prerequisites are known to be unsatisfied. If unable to 
determine whether the prerequisites of a suggestion are satisfied or not, Ani 
keeps the suggestion. Next those suggestions that conflict with others (found to 
be opposites in the relating step) are eliminated. If there are still too many 
suggestions left, Ani repeats the previous steps using the "best when" conditions 
f ^ of the suggestions rather than the prerequisites. Following that Ani removes 

those suggestions that are not related to at least one other suggestion. This is 
based on the idea that if many suggestions are in some sort of agreement then 
maybe they know what they are talking about. 1 Finally Ani eliminates multiple 
suggestions from the same source since it seems excessive to display four or 
five different events just to establish that, say, Cinderella is shy. Of course this 
should depend critically upon the length of the film being made and the 



jf\ 



I. This rule is an example of one of the many aesthetic biases built into Ani. It is a reflection of the 
idea of "parsimonious specification of complicated phenomenon" which is discussed in Section B 

"Aesthetic Systems" on page 133. 
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importance of the character and scene in question. 1 

Ani invokes these elimination methods to get the number of suggestions 
down to a reasonable number, Ani decides what a reasonable number is based 
upon the desired level of complexity and obviousness of the film and the length 
of the scene. Sometimes an elimination method is too effective and not enough 
suggestions are left over. In such a case its effects are undone and that 
method is skipped. 2 

This leaves a few suggestions as to what should be conveyed in the 
subscene. These are combined with anything that the original film description 
specified should be conveyed. Ani then gathers suggestions for how to convey 
these things and gets both suggestions for other things to convey and 
suggestions for what to display. The newly found things to convey are put 
through this process recursively until there are only display suggestions left. 
These display suggestions are then ranked according to each suggestion's 
strength, how well its prerequisites are met (suggestions whose prerequisites 
are not met are rejected), how well the "best when" conditions are met, how 
well the method can run considering how much time is available for this 



1. Making Ani sensitive to such factors is yet another improvement that should be made to Ani. 

2. This rule is typical of the sloppiness with which Ani was built. If an elimination method eliminates 
very many (but not too many) suggestions and the next elimination method eliminates just a few then 
Ani assumes it is the last one that is to blame. What is interesting is that Ani's performance is hurt 
only slightly by following such approximate rules. 
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subscene, how often this suggestion's method has been used in this film, and 
how much it contributes to the overall variety of the film. Percentile scores are 
computed for each of these factors. An overall score is computed by adding 
each of the factors weighed by the level of various global film descriptors. For 
example, the desired level of obviousness is used to weigh the scores for 
strength of suggestions, and the coherence level is used to weigh the 
prerequisite scores, the "best when" scores, and the "time fit" scores. Ani 
simply picks among the suggestions for conveying the same thing by choosing the 
one with the highest overall score.' 
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Section C. How a Method is Elaborated 

The choice of a display method does not end Ani's problems. Ani must fill out 
the method by specifying how long the activity should take, what modifications to 
the usual dynamics of the characters should be made and how many times any 
subevents should be repeated. And in the case where any of the participants 
of the event have yet to appear in this scene (or their location is so 
inappropriate that a film cut is necessary) then it is up to the display method to 
suggest appropriate initial locations for the characters. 



1. I apologize for the expediency of using numbers and scores heavily here. Clearly the choose should 
be based upon more symbolic reasoning 
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When a display method is run it generates constraints for the locations of 
the characters involved in the activity. For example, when running the method 
for moving a small way towards some goal Ani adds the constraint that the 
character be a large distance from the goal (so that moving a small way towards 
it is perceptible). If both the position of the character and the goal has yet to 
be decided, then the characters are placed at opposite ends of the screen. If 
one is fixed then the other is placed as far away as possible (and just barely 
off screen if it is the moving character). If both positions are already 
determined and if they are far nough apart then nothing happens. But if they 
are too close then Ani's only cinemagraphic technique, the film cut, is used. The 
"camera" cuts and the characters involved are repositioned. This decision to cut 
should ideally depend on whether this subscene and the previous can tolerate 
the passage of time or change of location that a cut implies, but does no! 
currently. 

The amount of time to be allocated for the displaying of an event is 
usually the amount of time that that event requests. Its request is typically 
based upon the dynamics, locations, and activities of the participants in the 
event. Modifications to a character's dynamics depend very much upon what is 
happening in the event. For example, if a character is trying to escape from 
another then that character's speed will be greater than normal and the path 
will be straight (even if the character typically moves in slow graceful curves). 
The number of repetitions of a subevent (such as the number of times that the 
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stepmother will hit Cinderella in the previous example) is based upon the 
activity level, rhythmic level, and repetitiveness 1 of the pacemaker of the event. 
The pacemaker of an event is the one whose movements are not in direct 
response to another. For example, in a chase scene in which the pursued is not 
to be caught nor to get too far away the pursued is the one who sets the pace 
and the pursuer acts only in response to the other's movements. In deciding 
how many times to repeat an event the estimate of how long the event will 
take is very important. The desired levels of obviousness and variety of the 
film are also taken into account. 

It is interesting to note how the values chosen for the dynamics of the 
characters influence Ani in choosing methods and activities. At the early stages 
of the decision process these values are used to evaluate the prerequisites and 
optimality conditions of the suggestions of what to convey. For example, a 
prerequisite of someone successfully chasing someone else is that they be faster 
than the other. Also the filling out of methods and the determination of how 
good a fit there is between the running time of a method and the time that was 
allocated for that subscene are influenced by the typical dynamics of the 
characters involved. 
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I. These are all dynamics elements whose values were chosen in the manner described in the previous 
chapter. 
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Ideally, there should be a playing off of the method choosing and the value 
choosing. If Ani is hard up for a method to establish something and the only 
available one was rejected because its prerequisites indicate that a character 
needs to be faster than another then this should cause the speeds of the 
characters involved to be redetermined. Or alternatively since Ani can make a 
character temporarily act atypically the default dynamics could be left alone and 
the characters could move unusually during this event. 

There is one unusual type of method that indicates no activity at all, only 
changes to the dynamics of a character. The changes can either indicate a new 
temporary value for an element or a factor to temporarily increase or decrease 
f\ the normal value by. The changes also indicate how fast the changes 

themselves should be. These changes are typically suggested for establishing 
the emotional state of a character. A sad character, for example, tends to move 
more lethargically and more directly. A happy character moves fast, accelerates 
quickly, and tends to move in a more bouncy manner. The problem with these 
suggestions is that they cannot stand alone. If nothing else is happening during 
the subscene in which these changes are to be displayed, then Ani tries to 
extend the length of the previous activity. It is during the extended part of the 
previous subscene that the dynamics of the characters changes. For example, in 
the "kept apart" scene the sadness of Cinderella and the joy of her stepmother 
at the end are conveyed by extending the previous activity of "standing guard". 
Ani attempts to convey this by having Cinderella's movements become more 
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sluggish while her stepmother movements become faster and bouncier. If Ani 
cannot extend the previous subscene then the character in question will "wander 
around" so that the changes are apparent. 

Section D How Multiple Activities are Ordered 

Ani would be simpler if every activity were allowed to run for as long it 
wanted. The problem is that Ani tries to constrain the film to be a (user) 
desired length. Ani must often arbitrate between the different activities by 
stretching or shortening them. Each activity responds to questions about how 
long it would ideally like to run for and also what is the minimum and maximum 
time that is acceptable. If a reallocation is necessary the activities themselves 
are asked to stretch or shrink. They do this by changing the speeds of the 
characters involved and by changing the number of times a subevent is 
repeated. 

An even more difficult problem than dividing the time between events is 
how to order unordered events. They can potentially be run in any order or 
simultaneously. The part of Ani that handles this is rather underdeveloped. Ani 
attempts to run the changes of the dynamics of the characters along with other 
events. Ani runs any unordered group of events in parallel if they have no 
participants in common. So if the stepmother and the Prince are to do 
something together and so are the fairy godmother and Cinderella, then Ani will 
run the two events in parallel. Even this is a bad idea if the film is supposed 
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to be simple or if there is more than enough time to do both sequentially. 

There is currently no mechanism for deciding the order of those events 
that Ani has decided to display sequentially. This problem occurs primarily when 
the scene description calls for many things to happen without specifying any 
order. After determining which events will be run simultaneously, Ani arbitrarily 
orders the remaining sequential events. To order them sensibly, Ani would need 
to know more about the pre-conditions and the post-conditions of an event. 
This problem of ordering events based on the pre-conditions and post-conditions 
has been addressed by many ([Sacerdoti 1975] and [Genesereth 1978], for 
example). 1 

The elaboration of a method and the selection of a method from a set of 
methods for accomplishing the same thing are both processes that do not 
interact very much with the selection and elaboration of other methods. This 
means that the processes can be run in parallel for each of the tasks that Ani 
is to achieve within a scene (actually the scenes could be worked on in parallel 
too). The potential parallelism in the selection and elaboration of events should 
not be confused with the real parallelism that occurs within a scene (e.g. having 
a few activities occur concurrently). Much of this potential parallelism would 



I. Ani typically does have available the pre conditions and post conditions for the locations of the 
characters but does not currently take them into account in event ordering. 
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disappear, however, were Ani more sophisticated. It was much simpler to build 
Ani to work on the selection and elaboration of activities without taking into 
account what happens in other scenes. This is one of Ani's bigger weaknesses. 

The interaction between scenes is limited even when there is an explicit 
connection between scenes such as in the third scene where the fairy 
godmother undoes whatever the stepmother did in the second scene that 
prevented Cinderella from meeting the Prince. The third scene is strongly 
influenced by the second but the reverse is not the case. In other words, in 
deciding what will happen in the second subscene, Ani does not take into 
account that whatever is chosen must be undone by the fairy godmother later. 
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Chapter VII Other Research 

it is difficult to find instances of research which relate directly to the problems of 
knowledge-based computer animation. It is easy to find relevant literature whose 
relationship is not so direct. Within Al there are knowledge-based systems for 
doing medicine, electronics, programming, chemistry and so on. The research on 
common sense reasoning is also apropos. Ani was put together with relational 
databases, actors, and small experts. There is much literature dealing with these 
concepts and their use ([Hewitt 1975b] and [Winston 1977]). Ani explicitly records 
the justification for most choices and occasionally inspects these records in making 
other decisions. There are systems like TMS [Doyle 1978] which are concerned 
with doing this is a clean and general fashion and systems which use records of 
their reasoning to explain their actions. SHRDLU [Winograd 1972] is an early 
example of such as system. Research on knowledge representation is very 
appropriate, especially the work on representing actions and activities such as 
Schank's work on scripts and plans [Schank 1977]. Al work on natural language, 
especially the story understanding work, is not only of theoretical interest to this 
research but could provide a user-oriented front end. Rieger's research on 
understanding the children's story book, "The Magic Grinder", is of interest, not only 
because of his representation of a fairy tale, but also because of his system's use 
of visual picture information and how it is related to the story [Rieger 1977]. 
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Several Al systems are concerned with planning and making choices in the 
presence of explicit constraints. Sacerdoti's work on non-linear hierarchical 
planning is relevant [Sacerdoti 1977]. His NOAH program has "critics" that look out 
for particular kinds of problems and then correct things. For example, there is a 
general critic that looks out for conflicts and resolves them. His work could have 
(and perhaps should have) been incorporated in the part of Ani that elaborates the 
chosen activities. Ani needs to plan the activities so that no prerequisites are 
broken, no superfluous movements occur (as they might if Ani did backtracking), and 
the characters are placed optimally. Ani manages to plan these activities but in a 
less general and reliable manner than NOAH. 



/"""N 



Both Ani and NOAH make their plans assuming they know the effects of all 
planned actions. McDermott built a system called DESI for designing electronic 
circuits [McDermott 1977]. A component of it called NASL does not make this 
assumption when planning. DESI works in a manner similar to Ani: working on 
subproblems,' adding new constraints for subsequent subproblems, and anticipating 
and handling conflicts. 1 



I I suspect that there are more points of similarity between Ani and DESI. My ignorance of 
electronics has prevented me from understanding' NASL as well as one might. 
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Anrs job is to move shapes in space through time. Al research on movement, 
shapes, space, and time has been done. Perception of motion has been the concern 
of many vision systems such as [Ullrnan 1978]. The description and recognition of 
shapes is a crucial aspect of much vision research. These systems can be viewed 
as performing the inverse of Ani — vision systems create high-level symbolic 
descriptions of objects and their motions from visual information while Ani creates a 
film (visual information) from a high-level description. Unfortunately, the current 
state of the art in machine vision does not provide many ways of representing or 
conceptualizing images and their changes that are much use to Ani. Ani could be 
improved by being connected to a vision system as discussed in Subsubsection a. 
"Lack of Perception" on page 228. The relationship of this research with expert 
systems for dealing with space [Kuipers 1977] or time [Kahn 1977c] is similar to 
the situation with vision — - the level in which they deal with space or time is not 
very relevant for Ani. 

Programs that compose music are potentially relevant to this research as 
another instance of computer creation of aesthetic objects. Also, the relationship of 
music and animation can be very close— some animation is even called "visual 
music". (See Section E "Animation and other Arts" on page 140.) Unfortunately, 
few such programs exists. Several programs make heavy use of random-number 
generators [Reichardt 1971], however, as discussed in Subsubsection b. "Creativity 
and Randomness" on page 212, this has little to do with creativity. Many other 
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programs are essentially aids for human composers to express their creativity at a 
relatively high level ([Truax 1977], for example) but do make choices of aesthetic 
consequence and use relatively little knowledge in their operation. 

This chapter does not contain a discussion of Talespin, a system that 
generates simple fables, or AM, a system that discovers simple mathematics. A 
comparison of these systems and Ani is very instructive and can found in Section F 
"Other Systems that Create Aesthetic Objects" on page 47. 

Section A Systems that Envision 

The research of Boberg [Boberg 1972] on generating line drawings of 
f^ three-dimensional objects from abstract descriptions is of interest. As a user 

describes a scene of blocks on a table the system, on the basis of reasonable 
defaults, incrementally fills in the details and draws (or redraws if necessary) the 
scene. The user can constrain the system's choices, for example, by stating that a 
particular block's position is not to be changed. Constraints also restrict the 
relative position of objects, so the user can state that block A must be always be 
to the right of block B. Heuristics for satisfying constraints are kept in a database 
and are applied to the current scene. Either when told to by the user or due to a 
lack of solutions of the current subproblem the system backtracks and undoes 
previous choices and continues with new ones. Boberg's system's use of knowledge 
and constraints resembles Ani; Ani however, is a much more ambitious project 



^""N 



Page 1 32 



o 



jf**^ 



/""*% 



Other Research Systems that Envision 

entailing both affective and aesthetic considerations and dynamics. 

Simmons [Simmons 1975] developed a system where one manipulates a 
graphically simple clown by giving natural language instructions. Superficially, it 
resembles this research; however, his emphasis is on the understanding of the 
natural language involved and the envisioning or description elaboration process that 
occurs in both Boberg's envisioner and Ani is lacking. Graphics is a good domain for 
doing natural language research because of its demand for details and the ease with 
which most mistakes are discovered (since the picture looks wrong). See [Kahn 
1975b] for an example of a pedagogic use of graphics as a domain for natural 
language understanding programs. 

Section B Aesthetic Systems 

Some work in the Al field superficially is very related to this research. For 
example, the work of Gips and Stiny on aesthetic systems and their application to 
"painting" ([Gips 1973], [Gips 1974] and [Stiny 1978]) at first glance would seem 
very related. Their approach is formal and based upon information theory. They 
describe a logical framework for interpretation and evaluation of aesthetic objects. 
The framework is general and applies to "painting, sculpture, literature, music, 
mathematics, and science" [Gips 1973], They define a mathematical system for 
interpretation and evaluation that provides a framework that guides the making of 
specific aesthetic theories. As an example, they apply their framework to 
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"paintings" generated (or describable) by a context-free grammar. It is the 
evaluative aspect of this that is potentially the most relevant to Ani. They define 
an aesthetic measure which is the ratio of the length of the description of an object 
to the length of the shortest description (with respect to some algorithm) for 
generating the object. In their "painting" example, this is the ratio of complexity of 
the painting (roughly the number of vertices) to the length of the minimum shape 
grammar that produces it. Given two "paintings" and the shape grammars that 
produced thern their measure can declare which one is more aesthetic. 

The process of making a formal general framework for aesthetics 
necessitates great simplification. More is involved in evaluation than the complexity 
of the product and the process that generated it. Aesthetic evaluation also entails 
perceptual factors (such as harmonious colors) that are not captured by information 
theoretic measures. Evaluation is concerned with the structure of the object, how 
its parts interact, how it is structured into levels, and how the levels interact. 
Affective aspects are also very important. (For example, the aesthetic evaluation 
of a swastika cannot depend solely on the complexity of the image and the minimal 
shape grammar for it.) Nonetheless there is something appealing in the idea that 
aesthetics is related to the "parsimonious specification of complicated phenomenon". 
They relate this to a long tradition beginning with Fechner's "unity" and "variety" 
and Birkhoff's "order" and "complexity" [Birkhoff 1933]. While intuitively the idea 
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that simple means should ideally generate complex ends seems right, 1 it is difficult 
to mirror this intuition in information-theoretic terms. This is because information 
theory does not capture very well what an art critic means when saying that an art 
object is complex, varied, or coherent. The psychological complexity of an object 
depends upon the language used to describe the object and the novelty of the 
object. This must be the case for any aesthetic measure, if it is to have any 
psychological validity. Since the evaluator of an aesthetic object has no direct 
access to a description of the process that generated the object it is difficult to 
apply Gips and Stiny's framework in practice. Also it is not clear whether the 
complexity of the process or algorithm that generates an art object is more 
appropriate or if the complexity of the trace of that algorithm as it makes the 
object is, 2 Complexity is clearly relevant to aesthetic evaluation. The problem that 
needs to be answered is with respect to what knowledge should the complexity be 
judged. 



1. One should always be careful when making "universal" aesthetic statements of any kind. The 
historv of art shows that many aesthetic criteria that our culture accepts without question were valued 
verv ciiffeiently in the past. Applying this idea of Gips and Stiny to contemporary art is difficult 
because of the modern emphasis on art that makes statements about art and so may intentionally break 
any rule. 

2. Bnkhoff in [Birkhoff 1 933] use the complexity of the trace of the viewer's perceptual process for 
the denominator of his formula of aesthetic measure. 
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I am not arguing against the idea of a formal theory of aesthetics. I am very 
skeptical that any theory which quickly reduces complex structures to simple 
quantitative values will be adequate for dealing with aesthetics. My bias is 
towards a theory that is based upon the formal structure of the complex 
descriptions of aesthetic objects and their relationship to other knowledge 
structures. The place of such theory for computer systems like Ani is discussed in 
Section F "Incorporating a General Theory of Aesthetics" on page 1 94. 

Section C Societies of Experts 

Ani is implemented as a large (currently about 500) community of small experts. 
f*^ Among the various experts are ones for the descriptors (e.g. "shy", "hates"), for 

elements with three ordered values (e.g. "low", "medium", "high"), for maintaining 
the desired variety level, for choosing relative values, for locations on the screen, 
for displaying actions, and many more. These experts, while simple, are more 
complex than rules or productions (see, for example, [Shortliffe 1976] and [Newell 
1972]) because they respond to a variety of messages (i.e. perform a variety of 
tasks) and because they maintain their own state (in their local databases and 
variables). One can organize these experts by very different control structures. 
One extreme is like Selfridge's "Pandemonium" system ([Selfridge 1 959] and 
[Lindsay 1977]) in which "demons" are constantly shouting what they think and 
whoever "shouts the loudest" is in control. The strength aspect of suggestions is 
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similar to this, but Ani chooses between suggestions based upon the relationship of 
the different things being suggested, the sources of the suggestions, and the 
number of sources that suggest essentially the same thing, in addition to the 
strength with which the suggestions were made. 

Ani's experts are built out of actors. Actors can be organized by any known 
control structure, ranging from completely decentralized independent modules to a 
strict hierarchy [Hewitt 1977a]. 1 Ani is implemented with a small top level that 
farms out tasks to others and resolves conflicts (with the help of other experts). 
Minsky and Papert in [Minsky draft] develop a theory of intelligence based upon a 
large number of interacting "agents" organized into different kinds of societies. 
/""N These societies are organized in many different ways but expertise and 

responsibility tends to be distributed so that no single agent is ever very smart. 
They describe subsocieties which are more flexibly organized than are most of Ani's 
experts (this is perhaps one of Ani's shortcomings). Despite many differences, one 
important theme runs through all of these systems and theories: that a society of 



I. Control structures that depend upon broadcasting to others are not so easily modeled by actors since 
each actor must know (either directly or indirectly) all the actors that they send messages to. See 
[Kornfeld 1979] for a description of a broadcasting model of computation that is complementary to the 
actor model. In Director, this is not a serious problem because of its convention that each actor know 
all of its descendants. Ani frequently does limited broadcasts to parts of the 
specialization generalization hierarchy. On the other hand. Director depends upon Lisp's recursive 
control structure and so certain control structures need to be simulated by making use of the actor's 
abilitv to maintain a state. 
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people is an extremely powerful metaphor for intelligent behavior (as opposed to 
the also powerful, but more conventional, model of intelligence as an individual). An 
important by-product of a society model of intelligence is that implementations 
based on it could efficiently use the multi-processor computers of the near future. 

Section D Relationship to other Computer Graphics Systems 

There are many computer graphics systems that make animation. Their relationship 
to Ani is tenuous however because they make no explicit choice of any 
consequence. The strongest connection is an implementational one — that the films 
that Ani creates need to be realized by a computer graphics system. This research 
/""\ is not concerned with how to display a square moving across the screen. Nor is it 

concerned with how to describe a square moving across a screen, even though Ani 
must eventually do that. 1 The essence of Ani is deciding what to 
describe — whether the square should move quickly from the upper left or slowly 
from the middle or whether it should chase a circle instead. 

Demonstrative animation system such as Genesys [Baecker 1969] or Shazam 
[Kay 1977a] provide a user with a means of graphically describing how various 
objects should move. Some demonstrative systems, such as the one at the National 
Research Council of Canada or the one at the New York Institute of Technology, are 



I. My work on Director is very concerned with these problems. 
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called in-betweening systems. They provide a user with the ability to draw the 
"key frames" directly into the computer and the system then produces a series of 
drawings between the "key frames" to smooth the transitions. ([Kahn 1977d] 
contains a survey of these systems.) Such systems are often very good but are 
limited to images and movements that can be described graphically by hand. A more 
important limitation results from such system's almost total lack of knowledge of the 
entities that they are animating. To achieve quality animation, knowledge is 
required so that the subparts of an image move naturally. Current in-betweening 
systems operate syntactically and as a result walking legs move strangely or an eye 
may split in two as an image goes from a profile to a direct view. The need for 
guidance by knowledge has been recognized by some of the developers of these 
systems [Catmull 1978]. 

The other approach taken to making computer animation is procedural. 
General purpose languages with graphic primitives such as Smalltalk ([Kay 1 977a] 
and [Goldberg 1 976]) and three-dimensional graphics systems such those at the 
University of Utah and the Jet Propulsion Lab are good examples ([Blinn 1976]). 
These procedural systems are similar to Ani in a few ways (indeed most of Ani's 
knowledge is procedural), but they make no choices of any consequence and their 
knowledge is in a much less general or flexible form. All decisions of consequence 
are programmed by the user for each program. Such systems are very relevant to 
the development of Director, the general purpose computer graphics language in 
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which Anrs programs are written. The difference between knowledge-based 
systems and such general-purpose systems is exemplified by DIAGRAMER as 
described in Appendix Vli. "A Knowledgeable Diagram Maker" on page 277. 

The Thinglab of Borning [Borning 1979] is interesting middle ground between 
Ani and the procedural graphics systems. He implemented in Smalltalk a graphical 
simulation laboratory. Rather than procedurally describe an object or activity, a 
Thinglab user declares constraints to the system. Thinglab then generates the code 
to satisfy the constraints. For example, one can describe a square to Thinglab as 
four equal length lines pairs of which are parallel. Both Thinglab and Ani expect the 
user to describe (as opposed to program or demonstrate) an activity. They differ 
/•""n greatly though in the level of abstraction of such descriptions. Thinglab is designed 

to function in a sufficiently constrained situation while Ani thrives on very 
under-constrained tasks. An interesting possibility is to use a system like Thinglab 
as the intermediary between Ani and a graphic display. 
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Section E Animation and other Arts 

That you can study in theory and in practice the craft of animation 
goes without saying. But by extending your curiosity to arts and 
sciences entirely foreign to your profession [animation], you will be led 
to the most unexpected insights. 

Alexander A lexeieff [Alexeieff 1969] p. 43 

This research is primarily in Al and computer graphics, but it is also animation 
research. This would not have been necessary if literature existed describing, at a 
low enough level of detail, what it is that animators know and how they plan and 
design their films. If there at least were some theories of animation, as there are 
theories of music and painting, then Ani could have been built upon them. Most of 
what is written about animation is anecdotal or technical. The few that attempt a 
theoretical discussion of animation borrow heavily from psychology' and the related 
arts of theatre, film, drawing, painting, dance and music. All of these fields are very 
relevant. Animation, though, is unique and needs its own theoretical underpinnings. 
Film and theatre lack the synthetic dynamics that is the essence of animation. Music 
and dance are dynamic and share with animation many aspects such as rhythm, beat, 
and temporal structures. Music deals with a different sensory modality and, despite 



I. For n discussion of the relationship of this research and perceptual psychology research see Section 
D "Ani as a Tool of a Perceptual Psychologist" on page 153. 
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the many parallels presented in [Whitney 1977], is very different from animation. 1 
Drawing and painting can only weakly suggest movement, in contrast to animation in 
which movement and change are the primary material an artist works with. Many 
very excellent animated films have few, if any, still frames that can stand alone 
aesthetically. 2 An extreme example of this is a film I made based on a dynamic 
version of Julesz ? s perception studies based upon random dot patterns [Julesz 
1 971]. The film consisted of regions of static random dots and regions of dynamic 
random dots (regenerated at regular intervals). The shapes and even the 
movements of the regions are perceptible even though any single frame consists of 
only randomly distributed dots. 

To build Ani, a theory, even if a crude one, of what animation is and how it is 
made was needed. The basic framework of Ani was influenced by art history and 
theory especially the writings of Gombrich ([Gornbrich 1961], [Gornbrich 1963], and 
[Gornbrich 1 972]} and psychology of art especially Arnheim ([Arnheim 1957] and 
[Arnheim 1969]). The influence is strongest in the way that the different elements 
or aspects of the object to be created are structured. Ani breaks up the dynamics 
into parts and each of those parts into elements that are the units of decision 
making. The problem of creating the dynamics for the characters of a film becomes 



1. i tints of Oscar Fischinger [Russett 1976] and parts of Disney's "Fantasia" are deliberate attempts to 
create visual music or a visual representation of music and as such might be considered exceptions. 
2 Full animation, such as that produced by the Disney studios, often does not fall in this class. 
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one of choosing a reasonable and self-consistent set of values for the elements of 
the dynamics of each. This is analogous to the description Gombrich and others give 
of the process of painting. Were Ani extended as described in Subsection 1 
"Adding Appearances to Ani" on page 163 to choose the shapes, sizes, colors, and 
textures (and the changes in these aspects) then each of these aspects could be 
broken down into elements and treated in a similar manner. New problems for Ani 
arise in handling the interactions between these different aspects. The literature 
on the history, theory, and psychology of art discusses this problem at length. 

Gornbrich's writings also give some clue as to why Ani's films work at all. It 
is easy to see simple shapes moving on a screen as people with goals, fears, 
/"""N feelings, and relationships. ([Michotte 1950] and [Heider 1944] present 

experimental evidence of this.) This is not very surprising if one accepts a "frame 
system" [Minsky 1975] or "schema" view of perception and recognition. Many have 
presented the idea that the process of perception is one of matching schemata to 
situations; Gombrich's special relevance is in his application of the idea to art. Once 
a system of schemata is "instantiated" one "sees" all the details that are present in 
the default schema unless overridden by the external stimulus. Compatible with 
this theory is the idea that people typically have schemata for the kinds of 
activities that Ani animates. Such schemata take effect when, for example, one 
sees a square chase and repeatedly hit a star. Once this happens it is difficult to 
see the shapes as "just shapes" moving around but instead one sees the scene as 
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someone hurting another. Understanding something of the perception of 
non-figurative dramatic animated films is very important in designing a system that 
makes them. Of even more help would be theories of this sort about the creation 
(as opposed to the perception and appreciation) of such objects. Unfortunately 
such theories do not seem to exist. 
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Chapter VIII Applications of Ani 

This chapter describes possible applications of systems similar to Ani. As a 
potential maker of interactive animation, some relevant uses of animation are briefly 
presented. Animation as a communication medium is discussed, especially how its 
role in communication might expand greatly due to computer technology. The 
educational applications of Ani are discussed both as a medium for learning about 
animation and as a producer of instructional films. The potential of 
knowledge-oriented computer animation for dynamically illustrating programs for 
documentation, instruction, and debugging is presented. The usefulness of Ani as a 
tool of perceptual psychologists is described. 

Section A Ani and Animation 

One of the most obvious uses of Ani is as a producer of animation. While the 
construction of knowledge-oriented systems like Ani is difficult, once they are 
operational they can produce animation much more easily than by hand or by 
conventional computer animation systems. The animation can be remade for 
different audiences or to keep up with changes in the subject matter of the films. 
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Animation as Art 

We do know that the cartoon film is an art as limitless as imagination 
itself. And within this immense sphere there is surely enough room for 
all kinds of emotion, for any ventures of man's mind, and not only for 
fun and entertainment. 

Dusan Vukotic. a Yugoslavian animator 

Ani was designed to be an experiment in building a computer system that 
could make animation as art (as opposed to animation that is primarily entertaining 
or instructional). If a more sophisticated version of Ani were constructed, perhaps 
as described in Section A "Making Ani Do More" on page 162, then one might 
^^ expect it to produce animated films that could serve the same functions as artistic 

animated films made by hurnans currently do. The films produced are likely to seem 
foreign since they were made by a very foreign intelligence. Human animators 
would not be replaced by systems like Ani, instead the animation domain will be 
stretched by the production of a "new" kind of film. The films will be different but 
not different for its own sake. They will differ because of the thinking that created 
them is different from a human animator. This will be the case for the foreseeable 
future despite the fact that such a system was designed to make its films in as 
human a way as possible. This improved Ani should be able to make films that not 
only people enjoy but that also stretches their minds a little. 
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Animation as Entertainment 

Animation is most often associated with children's cartoons. Animation is that, but it 
is often much more. (See Section G "Animation — What is it?" on page 55 for 
description of just what animation can be.) Entertaining animation for adults is 
popular (e.g. "Yellow Submarine", "Fritz the Cat", "Snow White", "Fantasia" and so 
on) but suffers from the high production costs. Many are engaged in trying to make 
computer systems to help alleviate this situation, as described in Section D 
"Relationship to other Computer Graphics Systems" on page 138 and [Kahn 1977d]. 
These attempts would be more successful if one could communicate with the 
computer systems at a high level of description. A knowledge-oriented approach is 
essential in the long run. Of use in the short term Ani is full of low-level animation 
descriptors that are higher than what is used in most computer animation systems. 
The way in which a character's movements are described is the best example. 
Objects can move according to a complex dynamics, have goals that are symbolic 
(e.g. a good distance from place A and place B), and can interact with others that 
come in their paths. The way the different activities which the characters are 
engaged in are described and coordinated should also be of value to doing computer 
graphics researchers interested in making commercial computer animation. Appendix 
II. "The Running of the Cinderella Film" on page 250 describes this aspect of Ani in 
greater detail. 
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Animation as a Language 

And therefore the activity of art is an exceedingly important activity, 
ju$t as important as the activity of speech, and just as universal. 

Leo Tolstoy [Tolstoy 1898] page 76 

Why not, indeed, let the basic signs be kinematic? Each "dynamic 
hieroglyph" could then truly "tell a story". A -dynamic display of 
dynamic hieroglyphs could represent hundreds or thousands of 
interactions among tens or hundreds of ideas. To the untrained eye 
such a display would no doubt look like a bucket of worms, but to the 
trained eye...? 

J.C.R. Licklider [Licklider 1976] page 92 

Animation is a medium of communication and expression that is at least as effective 
as natural language (e.g. English). Natural language is, of course, much easier and 
cheaper to use and is therefore much more common. Animation and natural language 
do not compete typically — they are each most appropriate in different domains. 
The significance of this research is increased by the acceptance of this viewpoint. 
No one in the Al community questions the value of research on natural language, so 
for many of the same reasons they should accept research on the understanding and 
production of animation. 
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Many of the reasons why animation is much iess prevalent than natural 
language concern the difficulties of producing animation. Understanding animation is 
not considered a serious problem for humans, indeed many consider it so easy that 
it is only appropriate for children. Animation might become a very prevalent 
medium of communication and expression if it could be produced with as much ease 
as, say, typing a paper. You type to a futuristic Ani what you mean to animate 
(instead of typing what you mean to say) and out pops the animation to be seen by 
yourself or viewers. Licklider [Licklider 1976] may have been thinking along these 
lines when he suggested that writing in the future might be like animated 
hieroglyphics. 

/"•v Section B Ani and Education 

The educational uses of an Al-oriented computer animation system were important 
in the design of an early predecessor of Ani [Kahn 1976a]. The idea was that a 
properly designed system could be an ideal environment for a student to learn 
about both Al and animation. This design criterion was dropped for practical 
reasons in the making of Ani. Nonetheless, Ani could be viewed as a paradigm of 
how a skill could be taught to humans by conveying how that skill could be 
embedded in a machine. Since Ani is a manipulate glass box, a student of animation 
could learn much by observing the effects of altering Ani's knowledge and rules. 
Ani's knowledge is in an explicit form that can be studied in ways that the 
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knowledge of a human animator cannot. A student could also replace Ani's 
knowledge base with his or her own knowledge and see how a very crude 
simulation of themselves as animators performs. 

Ani is designed to produce entertaining aesthetic films but another system 
could be built upon similar principles whose purpose is to produce instructional films 
or dynamic diagrams. Such a system would know enough about pedagogy that it 
could produce its films with relatively little interaction with humans. In the 
not-so-distant future, one might expect an educational version of Ani to produce 
films on the fly to help intelligent tutoring systems such as those described in 
[Goldstein 1977]. If a picture is worth a thousand words, what is a short animated 
film consisting of thousands of pictures worth? 

Section C Illustrating Programs 

Animation, music, and programming can be thought of as different 
sensory views of dynamic processes. 

-Alan Kay [Kay 1977] page 35 

One exciting possible application of this research is the construction of a system 
that could animate programs. One could see a visual trace of a program in 
operation for debugging, or an animated metaphor of an algorithm in operation for 
teaching people what it does and how. One could interactively "fly" through the 
database of a complex program, "diving" into what is of interest, sailing quickly over 
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other parts. The animation can be tailored to a simple model of the viewer. A 
programmer can change a program and a new film of its operation can be 
automatically generated. 

It has long been known that visual presentation is a great aid to learning 
about, understanding, and thinking about complex processes. An animated dynamic 
presentation is of even more value. An example is the work of Baecker and his 
colleagues at the University of Toronto [Baecker 1978]. They have produced two 
different types of animated films of programs. The first were programs in execution 
where one visually sees stacks pushed and popped, variables bound, functions 
called (even recursively) and so on. These films were generated from the programs 
ffm^ themselves and terse illustration specifications. Their effectiveness was limited, 

however, because of their heavy use of textual information and their extreme 
graphic simplicity. The other films made at Toronto were much more effective and 
visually interesting. Nine different sorting algorithms were visualized as factory 
machinery with the data being moved along on conveyor belts frequently being 
operated upon by machines as they passed by. Unfortunately these films were 
hand-crafted in the sense that each film had to be explicitly programmed at a 
relatively low-level. Clearly what would be desirable would be a system that could 
produce films with the ease of the Logo program visualizer and of the quality and 
effectiveness of the sorting and hashing films. 
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What is needed to make such a program visualizer? For one, the kinds of 
knowledge and expertise that is in a system like Ani. The system will need to make 
many choices ranging from what visual metaphor to use, to how to graphically 
represent the objects and processes of the program, to where to place a particular 
component and how big it should be. In order to animate a program one needs to 
understand it. It is much too ambitious a project for the visualizer to automatically 
accomplish this, so the system would need to be provided an annotated description 
of the program and its organization. This is analogous to the situation where Ani 
applies general animation knowledge to a user-provided film description to make 
films. The program visualizer would apply its knowledge of animation and 
programming to a user-provided program description. 

The actor philosophy of computation can play a significant role in program 
illustration, it provides a good bottom level dynamic graphic representation of 
computation as actors sending and receiving messages. The actor event diagrams 
developed by Hewitt is a good start. A major difficulty with them is that they are 
trying to represent concurrent dynamic processes with static drawings. Real-time 
interactive animation would alleviate many of the problems involved in showing how 
a complex program, especially a parallel one, works. 1 



I A program need not be written in an actor language for this graphic representation to be of value. 
Ninny system programs, network programs, or most programs with a high degree of modularity can be 
profitably viewed as consisting of separate entities that pass messages between themselves. 
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Section D Ani as a Tool of a Perceptual Psychologist 

Ani can be viewed as a producer of material for perceptual psychology. The 
perception of dynamic qualities of a scene such as causality, emotions, 
inter-personal interactions, and activities is an important ability of humans that 
warrants study. 1 Ani is an ideal source of stimuli for such experiments. Ani can 
generate episodes which are presented to subjects to find out what are the critical 
parameters in a motion to produce a particular characterization, mood, or interaction. 

One ability of Ani is to produce minor variants of the same film. A study 
based upon these variations would be prohibitively expensive by almost any other 
means. A system like Ani enables one to control not only low level details such as 
the exact size of a particular character but also high level parameters such as the 
kind of rhythm or the relationship between two characters. A symbiosis is possible 
where Ani's knowledge gets refined by psychological experimentation and the 
psychological experimentation is aided by an excellent source of stimuli. For 
example, a psychologist might be interested in what dynamics evoke the idea that 
an object is afraid. Subjects could view films of characters acting afraid made by 
Ani and the findings can be used to refine Ani's methods and suggestions. New (and 



I (Michotte 1950]. [Michotte 196?]. [Weir 19753- and [Heider 1944] are the only instances of such 
studies that I am aware of. 
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presumably more effective) films can then be made and the process repeated. 
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Chapter IX Where Ani's Suggestions Come From 

Where do the suggestions that Ani manipulates come from? One answer is the 
descriptors and actions in the film description. Another is that they come from me, 
the implernentor. But where did I get them from? I did not just make them up on 
some arbitrary basis. Suggestions are a particular use of generalities (e.g., that 
most happy people move quickly suggests that to convey that something is happy 
make it move quickly). Suggestions come from observations of the real world, of 
people (and animals) moving, feeling, interacting and so on. Some of the behavior is 
of biological origin (e.g., fleeing or assuming a threatening pose in response to 
danger). Others are psychological in nature — related to how we perceive 
movements and actions. A good deal of it is just common sense. E.g., one wants, in 
general, to be with the ones they love. A shy person tends to avoid others 
especially those he or she does not know and so on. Good animators are good 
observers of behavior. They notice what movements convey and how they are 
used to express. This knowledge is then used to create artificial stimuli that evoke 
similar (or even stronger) responses from viewers.' 



I. Animators often create super normal stimuli, that is by exaggerating the right elements the stimulus 
can be moie realistic than reality. For example, a bouncing ball looks more realistic if it is flattened 
excessively when it hits the ground. As a matter of fact, it looks more convincing if the ball flattens 
just before hitting the ground Nonetheless, it helps to know reality in order to exaggerate or distort it. 
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Section A Etiological Sources 

We know that every strong sensation, emotion, or 
excitement— extreme pain, rage, terror, joy or passion of love--all 
have a special tendency to cause the muscles to tremble; 

Charles Darwin [Darwin 1872] page 217 



Most of our emotions are so closely connected with their expression, 
that they hardly exist if the body remains passive— the nature of the 
expression depending in chief part on the nature of the actions which 
have been habitually performed under this particular state of the mind. 

- Charles Darwin [Darwin 1872] page 237 



it is very interesting to notice how these requirements of simplicity, 
conspicuousness and specificity have been compromised by selection in 
exactly the same way as in signal structures developed by man by 
purposeful and conscious designing. This has been emphasized by 
Lorenz, who pointed out the striking similarity of visual releasers, like 
the wing-specula of ducks, and national flags. 

-Niko Tinbergen [Tinbergen 1951] page 184 

Some of Ani's suggestions have an ethological basis, that is they have their basis in 
animal (and human) behavior. If an animal suddenly runs away from another during a 
fight then one can safely assume that the animal was afraid of the other. Ani, for 
instance, has three methods for establishing that one character dominates another. 
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Ani can either have the character push the other around, have the character 
prevent the other from doing something that it wants, or have the character make 
the other do some activity that the dominating character wants done. A version of 
Ani extended as described in Section A "Making Ani Do More" on page 162, could 
have the dominating character typically appear above the other, do things first as a 
rule, or be larger than the other. Most of these methods are the type of clues that 
observers of animal or human behavior look for to understand the social rankings 
within, say, a baboon troop. Ethology is an excellent source of material on the 
expression of emotions and relationships by facial expression, body posture, and 
movements. An interesting presentation of this can be found in Darwin's 
"Expression of Emotions in Man and Animals" written long before the field was 
named "ethology". [Darwin 1872]. 
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Section B Perceptual Sources 

While we were doing these experiments we noticed a rather peculiar 
fact, which at first we considered simply rather curious, but which was 
repeated so often and with such insistence that it became a factor of 
some importance to us, and one worthy of serious discussion. Our 
subjects did not content themselves with merely describing in an 
objective fashion what they saw in the apparatus, saying, for example, 
that they saw "A pushing B forward," but they often had an obvious 
tendency to complete these indications by comparisons with human or 
animal actions, comparisons which implied emotional states, attitudes, 
tendencies attributed to the objects. The letters A and B did not then 
signify the little rectangles as such, but took on the value of names of 
persons, and the experiments gave rise to interpretations of this 
nature: "It is as though B was afraid when A approached, and ran off"; 
or "A joins B, then they fail out, have a quarrel, and B goes off by 
himself"; or again "It is like a cat coming up to a mouse and suddenly 
springing on it and carrying it off. " 

Albert Michotte [Michotte 1950] p. 115 

Some of the suggestions have their basis in human perception, especially the 
perception of causality. Michotte, in [Michotte 1963], provides many examples of 
how the causality of very simple events is perceived differently depending upon 
the exact timing of the events. He buiit a special-purpose mechanical device for 
displaying moving colored rectangles. This was used to show a precisely controlled 
sequence of movements to subjects. Subjects were then tested and interviewed. 
A typical finding was that observers of a square moving until it came in contact with 
another and then after a pause moving off together will describe the event very 
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differently depending upon the length of that pause. If the pause is very short then 
the square pushes the other. If it is long then it meets the other and they 
accompany each other. In the first case, but not in the second, observers perceived 
the first square as causing the other to move. His experiments are very relevant 
to the construction of many of Ani's suggestion sources and display methods. 1 
Michotte believed that he was studying an innate mechanism for directly perceiving 
causality. Regardless of whether his results are due to some innate perception or a 
learned high-level activity, they point to a real phenomenon. See [Weir 1975] for 
a good discussion of this aspect of Michotte's research. 

Michotte earlier did some work on the perception of affective qualities of 
very simple stimuli [Michotte 1950]. He showed rectangles moving to subjects and 
found that the emotional interpretations that they gave was very specific and were 
in general agreement with each other. For example, if A moves alone and contacts 
B then if it stayed and the approach was rapid, people would describe it as a 
violent clash and the two objects becoming welded together. He found that rapid 
movement gives an impression of violence while slow motion conveyed gentleness. 
Sudden and repeated changes in direction or even speed gave the impression of 
"nervousness" or "agitation". This kind of psychological research could provide a 
strong theoretical basis for much of Ani's knowledge were it not so scarce. To my 



1. I confess that i only loosely based Ani's knowledge on his data. 
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knowledge Michotte did only what he reported in [Michotte 1950] and that only 
Weir [Weir 1975] followed up in any way. 

Heider did similar ,/ork earlier [Heider 1944]. He showed a three minute 
"cut-out" animated film (made by filming stop-action a few cut-out pieces of paper) 
of two moving triangles and small circle to subjects. He found much agreement in 
his subjects" descriptions of the action as a love triangle in which the larger triangle 
is mean and attempts to come between the other two. 

One of the stumbling blocks to doing this kind of research is the difficulty of 
producing the stimuli. The special purpose mechanism constructed by Michotte was 
too limited and the hand-made animation of Heider was too expensive and 
time-consuming to make. Ani, with an ability to produce many variations upon a 
theme, is potentially a solution to this difficulty. See Section D "Ani as a Tool of a 
Perceptual Psychologist" on page 153 for a discussion of this application of Ani. 

Section C Social Conventions and Common Sense 

Some of Ani's suggestions are based upon social conventions. For example, Ani has 
two ways of establishing that a character is "good". The character can prevent 
something that it considers bad from occurring or it can help some other character 
who preferably should be weak but good. These are based more upon the 
conventions of our society than upon ethology or common sense. The high level of 
abstraction leads one to think of these as universally valid ways of establishing that 
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someone is good. Even with its high level of abstraction, much of Ani's knowledge 
is based upon social conventions. For example, one can imagine societies where 
someone who is good does not interfere even if bad things are happening. The role 
of social conventions is greater for appearances and body gestures than motions 
— "bad guys wear black", "smooth is friendly", "green is jealous", "a smile is 
happy", and so on. 

Many of Ani's suggestion are just common sense. For example, to prevent 
someone from meeting another one can either prevent the character from getting to 
the other or make the other no longer available. To establish that a character is 
shy one can have that character avoid others. To convey that a character is 
graceful one can move it in long slow curves. There is no strong distinction 
between these common sense suggestions and the social convention ones. It is only 
that the common sense suggestions seem less arbitrary. They are the ones that are 
hard to imagine how they could be otherwise. 
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Chapter X Improvements and Extensions 

This chapter discusses some future directions for this research. It contains most of 
the confessions about how little Ani really does, but counters them with ideas for 
improvements to Anj and sketches of how many of the deficiencies might be 
removed. Presented are ways that Ani could be extended to do more and still 
operate in basically the same manner. Minimal changes to Ani for producing better 
films are then discussed. Speculations about major changes to the structure and 
organization to make Ani "smarter" are then given. This chapter is necessarily 
sketchy; its purpose is to provide direction for further research. 

f*\, Section A Making Ani Do More 

Ani can't do very much currently. Ani can make very simple shapes move in order 
to convey some very abstract activities, affective states, and inter-personal 
relationships but can do little else. Ani is limited for good reasons — - primarily so 
that this research could be based upon a running implementation that was not the 
result - of a large team of programmers and animators working for many years. Light 
was shed upon many interesting issues and problems by constructing such a limited 
animator. Ani demonstrates that at least a littie bit can be done, but what if one 
wants more? 
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Adding Appearances to Ani 

One glaring deficiency of Ani is an inability to draw. Even the squares, triangles, 
and stars of the Cinderella film were not conceived or chosen by Ani. In [Kahn 
1977b] I outline how a system such as Ani would be able to intelligently determine 
the appearances of the characters. In much the same way that the typical dynamics 
of characters is handled, the appearance of a character would be broken down into 
aspects and the aspects in turn broken down into elements. The aspects of an 
appearance are the size, color, line, texture, and shape of a character. The 
elements of the color, for example, are the intensity, hue, and saturation. 1 The 
shape of a character is described as either smooth or jagged, long or short, closed 
or open, symmetrical or asymmetrical, simple or complex, and so on. The values of 
these elements would be chosen by an extended Ani in much the same way that the 
values of the elements of a character's dynamics are currently chosen as is 
described in Chapter V "Choosing Values" on page 103. 



I. Other ways of structuring the space of colors exist such as the relative proportions of red, green, 
and blue, however the intensity, hue. and saturation space is conceptually easier to deal with. 
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New problems arise in actually constructing a shape for a character after the 
elements of the shape description have been chosen. Somehow a particular shape 
such as a star or snowflake needs to be formulated from a description such as 
"moderately complex, regular, symmetrical and closed". To do this I envision a 
hierarchy of shapes where distributed throughout is the knowledge of how to 
construct a shape satisfying the constraints of the description. Near the top are 
experts corresponding to classes of shapes such as polygons, spirals, blobs, stars 
and the like. Each is asked if they have any suggestions for how to draw an object 
satisfying the description. If more than one suggestion results then they need to be 
combined, compared, and selected between on the basis of the desired levels of 
/""N. variety, simplicity, originality, etc.. If no shape class can satisfy the constraints then 

many remedies are possible. A constraint can be eliminated or the least justified 
value of the elements (that it be symmetrical, for example) can be redetermined 
(effectively changing one of the constraints). Ani would then try again with the 
modified description. If the problem persists, the "constraint relaxation" or 
"constraint redetermination" methods can be applied iteratively. This approach to 
symbolic constraint satisfaction is based upon the existence of a large database 
rather than a very smart system that figures out a shape. The restriction to 
non-figurative images and the structuring of the shapes into a hierarchy make this 
knowledge-oriented approach feasible. 
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Currently, Ani uses only one aspect of the characters (namely their dynamics) 
to realize their personality and mood If extended to use the color, shape, size, 
texture and line, in addition to the dynamics, of the characters, then new choices 
and new problems arise. Ani must determine which aspects to use to achieve some 
effect. If the film is supposed to be obvious, then many aspects should be 
used — if subtle, then only a few. Another problem is to handle the interactions 
between the different aspects. If, for example, the size is used to convey the 
strength of a character and the color and texture to convey a calm mood, then Ani 
should be careful that the interference between the two aspects is minimized. The 
same color in a small region will have different effects in a larger region, for 
instance. 

Changing Appearances 

Once Ani is extended to determine the appearances of the characters of the films, 
another extension follows naturally. The essence of animation is change and Ani's 
forte is the change of location of the characters (i.e. moving them). Ideally an 
extension to Ani to deal with appearances would include the ability to deal with 
changes in these appearances. Ani should be able to determine a character's 
appearance before and after a change in the same manner as described in the 
previous section. The new problems for Ani, if extended in this manner, are 
deciding when to change a character's appearance and how the transition should be 
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made. The first problem could be handled in much the same manner that Ani 
currently decides when to change the typical dynamics of character. (See Section C 
"How a Method is Elaborated" on page 122 for a description of how the dynamics of 
a character is changed.) A dynamics change is a display method that is not treated 
very differently from the display methods for activities such as hitting, pushing, or 
chasing. For example, a suggestion for how to establish that a character is very 
happy is that the intensity of the colors of the character be increased. The 
appearance can also be changed as part of a display method, for example, after two 
characters come in contact they might change their shape slightly to convey the idea 
that they are communicating. Not all appearance changes are internally generated. 
Sometimes the script explicitly calls for an appearance change, as in the scene 
where the fairy godmother changes Cinderella from being shabby and ragged to 
being beautiful and elegant. 1 



After determining that a character's appearance will change, Ani still has the 
problem of determining how this transition should be made. Occasionally the 
transition should be instantaneous, i.e. in one frame the character has one 
appearance and in the next frame a different appearance. More typically the 
transition should be gradual. A gradual transition can be displayed using 



I. This scene was deleted from the test of Ani since it is meaningless until Ani is extended to handle 
appearances. 
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interpolation routines. Interpolation is a very common technique of animators. They 
usually call it "in-betweening". The difficulty is that there are many ways to change 
an appearance gradually into another and that after determining the interpolation 
path there still remains the interpolation dynamics, e.g., the speed the transition 
should occur at. Director, the system runs Ani's films, has a general interpolation 
mechanism that preserves many properties (such as symmetries, connectedness, and 
openness) of the initial and final appearances while making the transition. Since the 
appearances are non-figurative Director's interpolation capability is adequate. 

If the appearances were figurative then the problem becomes much more 
difficult. A smile does not change to a frown according to some simple mathematical 
routine, but changes in accordance with a set of complex physiological constraints. 
Also once the images become figurative, the viewer will usually perceive the 
appearances as projections of three-dimensional objects and as such expect some 
transitions be rotations out of the picture plane. This problem with 
interpolation-based animation is discussed in [Catmull 1978] and entails many 
extensions to Ani some of which are discussed below. 
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Changing Descriptions 

The essence of animation is change. Ani currently can only change the location and 
orientation of a character. Ani portrays changes in the emotional states of the 
characters, as when Cinderella becomes very happy after finally meeting the Prince. 
The ability to change the appearance of a character can be added as described 
above. But what about changing the descriptions of the characters themselves? 
Many stories have characters who change more than their appearances. Shy 
characters learn to socialize, evil characters repent and learn to lead a "good" life, 
Pinocchio learns not to lie, and so on. How would Ani need to be extended in order 
to animate such stories? 

Before working on the scenes, Ani compares and contrasts the character 
descriptions and determines the typical dynamics (and were Ani extended as above, 
the appearances) of the characters. The personality, the role in the story, and the 
movement descriptors of the characters are assumed to be permanent. Of course it 
would be a minor change to Ani to recompute the elaborated descriptions of the 
characters at any point. If the personality of a character is supposed to change in 
the middle of a scene, then Ani could simply rederive the dynamics (and 
appearance) of the character with the new personality as if the character were 
brand new. This is not an ideal solution since Ani would not portray the personality 
change itself, the character would only move and appear differently after the 
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change. Occasionally this is just what is desired, but more typically the transition 
should be highlighted or gradual. Ani could be given the ability to compare the 
dynamics and appearance of a character before and after a change and then slowly 
bridge the gap between them. For example, if a character loses its shyness then it 
may move faster and more deliberately than previously. Rather than abruptly 
increase the character's speed (which could easily be misinterpreted as meaning 
something else) Ani would decide how long it should take to gradually increase the 
speed to the new value. In addition, Ani could have method suggestions for 
conveying a personality change that would work essentially the same as the 
currently existing method suggestions for establishing the personalities, emotional 
states, relationships and interactions of the characters. A suggestion for conveying 
that, for example, a character becomes strong would indicate that the speed be 
increased and at what rate and might even suggest an activity such as pushing a 
large object a long distance. 

Controlling the Camera 

As a cinematographer Ani is pretty poor. Ani makes cuts if the characters need to 
be drastically rearranged but never moves the "camera". While an animator films 
his or her images with a real camera (which is often stationary) there is almost 
always an animated camera or viewpoint that zooms, pans, and comes in for close 
ups. Sometimes the animated drawings simulate even more esoteric cinemagraphic 
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Controlling the Camera Improvements and Extensions 

techniques such as dissolves (the gradual fading out of one image concurrent with 
the gradual fading in of another), wipes (one image gradually moving over and 
obscuring another), change in the depth of focus (e.g. making objects in the 
background blur as objects in the foreground go into focus), double exposures and 
the like. Animators use such techniques while Ani cannot even cut to a close up to 
display two characters coming in contact. 

How could Ani be extended to use these techniques in a reasonable manner? 
The simplest extension would be to imbed suggestions for their use in the method 
suggestions. A suggestion for how to have someone hurt someone else that is 
currently in Ani is to have that character move close to the other and then 
repeatedly hit the other. This suggestion could be changed to move the camera in 
close after the characters gets near each other so that the hitting episode wilt be 
seen close up. A more major extension would be for Ani to "notice" where the 
action was in a scene and apply a few simple heurstics. For example, if the action 
is not localized in a small part of the screen then move the camera back, if the 
action is localized and way off center then pan until it is more centered, and so on. 

Regardless of the source of a suggestion that Ani move the "camera" many 
changes would be needed to elaborate and execute it. Ani would need to be able 
to decide how exactly the camera should move. If a method suggests that the 
camera be moved inward then many details remain to be decided. Should it zoom in, 
and if so, quickly or slowly? Should it simply cut to a close-up? How close should 
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the close-up be 9 If it zooms in, should it pan towards the center of activity at the 
same time (and at the .same rate)? These details of the description of the camera 
movement would depend upon many of the same factors that are currently used in 
elaborating methods. If the amount of time allocated to the scene is short then a 
cut or quick zoom is appropriate, if it is long then a slow zoom is possible. If the 
tempo and energy of the film is high then a cut or a very fast zoom is ideal. If the 
desired level of variety of the film is high, this scene is to stand out, and most of 
the previous camera movements were cuts to a medium distance, then a zoom to a 
close distance is indicated. If something else is happening concurrently on a 
different part of the screen then the situation is more complicated. Either the 
,/***N camera movement should be called off or else cutting back and forth between the 

two events is called for. 

Much of the knowledge needed to extend Ani to be able to control a camera 
is just common sense. If activities are happening all over, then pull back and let the 
viewer see it all. If all the activities are restricted to a small part of the screen, 
then move in on the area so the viewer can see it better. The remaining 
knowledge needed can be found in the film and animation literature. Much of that 
literature, however, in concerned with much more subtle aspects than the discussion 
here. A cut might be motivated by the need to best show what is going on, but it 
should be constrained by many aesthetic considerations. The frame before and 
after the cut should not be too similar. The images should either complement, 
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contrast, or parallel each other. Ideally this should be true of the images, in term of 
the colors, the balance, the textures and also true of the dynamics before and after 
the cut — the movements before and after should relate. The timing of the cut is 
very important. An episode or scene should finish cleanly, typically by cutting at 
some natural pause in the activity. Following such rules brings in another level of 
difficulty that is discussed further in Subsection 7 "Non-Narrative Animation" on 
page 1 76. 

Extending her Domain of Expertise 

Ani's films are lacking in many aspects that are commonly found in hand-made 
animation. The lack of scenery or backgrounds is a good example. Ani only 
animates characters and has no conception of stationary objects. This could easily 
be partially remedied by creating characters and then never have them do anything. 
The real problems are deciding what scenery should exist and where it should be 
placed. Scenery or backgrounds perform such functions as providing a frame of 
reference for the nearby action, enabling identification of different places 
(especially important if the "camera" were able to move as described in the 
previous section), helping to establish a mood or setting for a scene, or making the 
scenes visually more attractive. Each of these functions suggest different kinds of 
scenery and provide some clues as to what the scenery should be look like. For 
example, if the scenery is to provide a frame of reference or identify a location 
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then it should be distinctive, not centrally located, and not interfere with the actions 
in the scene. Scenery us*-"' to establish a mood or setting should be distributed and 
pervasive. If it is there to pretty up a scene then its placement and the balance of 
the colors and shapes involved become more important. At the limit, the ability to 
handle backgrounds effectively is equivalent to the ability to paint or draw a 
picture — - a useful and very important ability, but a different problem from making 
animation. 

Consideration of the question of how the characters of a film should interact 
with the background scenery leads to the more general deficiency of Ani, namely a 
two dimensional way of dealing with images. Ani has no conception of a character 
being in front of or behind another, If a character happens to cross over another 
then it is arbitrary who will be drawn on top. 1 A small extension to alleviate this 
problem ; ~ to give Ani the ability to determine and describe a partial ordering of 
the objects on the screen. For example, a rule stating that if a character is 
dominating another then it should typically be drawn on top of the other could be 
added. This is similar to what is called "2 and 1/2 D" in computer graphics, since 
the third dimension is represented as a partially ordered list of planes. 
Conventional eel animation, with its layers of drawings on clear acetate, is 



1. The participants of an activity often are told to avoid contact with others, but if not. then collisions 
do occur. 
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sometimes used to create 2 and 1/2 D this way. 

The next step is what is sometimes called, "full 2 and 1/2 D" or "2 and 3/4 
D", and still involves a list of planes but now the planes are at different distances 
from the camera thereby creating a greater sense of depth since objects further 
away appear smaller and move slower. Some computer graphics systems are built 
upon this scheme [Levoy 1977], as was Disney's multi-plane camera which was 
used in many of their animated films twenty or thirty years ago.' 

To make films this way, Ani would need to decide which plane each object 
was on and how far apart the planes were. From this it is only a small step to full 
three-dimensional animation of characters which are flat and parallel to the plane of 
the "camera". Ani could choose locations and trajectories in much the same way 
that they are now chosen in two dimensions. The next step would be for the 
characters themselves to have three-dimensional appearances and again no new 
theoretical problems arise in going from two dimensional to three-dimensional 
animation. 2 



1 The multi plane camera was generally acknowledged to be an advance in the production of quality 
animation but was prohibitively expensive to use and was mostly phased out. 

2. Of course. Director would have to be greatly extended to display perspective drawings or shaded 
images of these three dimensional scenes. Much of the computer graphics literature addresses this very 
problem ([Blinn 1976] and [Newman 1973] for example) and there are no technical reasons why this 
could not be done 
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Figurative Animation 

The animation that Ani makes is unusual It is intended to "tell" a story but the 
appearances of the characters are not figurative — they don't look like Mickey 
Mouse or Bugs Bunny. To make animated films with figurative images requires an 
ability to draw of the same caliber as the ability to animate (that is to control the 
movements and changes of the images whatever they be). It is interesting to 
contemplate whether the same general approach embodied in Ani for making 
animation could be applied in constructing a system capable of making drawings in 
response to vague and incomplete descriptions. It is a simplification to assume that 
the ability to draw can be so easily separated from the ability to animate. The way 
f*^, that images move depends very much upon what the images represent. The 

inverse is also true though less obvious; the way an image is rendered is influenced 
strongly by how it is to change. (For example, a subtle color effect in a drawing of 
a character will be overridden if that character moves quickly.) 

Figurative animation demands more than the ability to draw and to animate; it 
demands that the animator be able to draw upon a great deal of real-world 
knowledge. Ani's films are like what one might see from the top of a mountain 
looking down at a field with a few people in it. Their gross movements are visible, 
but their body parts and faces are indistinct. From such a view one has no 
expectations of seeing characters shaking hands when meeting, frowning when 
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worried, running by moving and bending their legs in a particular complex pattern 
and so on. Figurative animation presents a close up view and as such a viewer 
expects activities such as these to happen in a reasonable manner. To do this Ani 
would need to know much about the real world and common sense. It is easy to 
see how any missing fact, for example, that people often raise a hand in greeting 
can be added to Ani so that it will be used when a greeting takes place. Indeed, 
[Kahn 1 976a] is a design for such a real-world knowledgeable animator. While we 
can imagine how any particular needed fact can be incorporated into Ani, the 
problems arise when one tries to imagine how to incorporate in some general and 
systematic way all the facts that Ani would need to know to animate, say, the real 
f\ story of Cinderella complete with pumpkins, glass slippers, and tattered clothes. 

Giving computers some common sense is a well-known problem in Al (see [Minsky 
1975] and [Schank 1977], for example) and may prove to be the major theoretical 
obstacle to extending Ani to make figurative animation. 

Non-Narrative Animation 

My computer program is like a piano. I could continue to use it 
creatively all my life. 

- John Whitney. Sr. [Youngblood 1970] page 207 
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The essentia! problem with my kind of graphics must resemble the 
creative problem of melody writing. It is perhaps the most highly 
sensitive task of art, involving as it does balance, contrast, tension, and 
resolution all brought into play with minimum expenditure. 

John Whitney, Sr. [Youngblood 1970] page 220 



Animators have discovered the possibilities of articulating line and 
space, of creating objects of meditation, of presenting synchronization 
of visual rhythms to music, of making abstractions based on concrete 
objects or experiences, of telling stories — through a variety of 
methods. 

As new advances in technology are made, it is certain that animators 
will experiment with these techniques, creating new forms and 
/*"% definitions of animation. 

[Whitney Nfuseum 1976] Page 73 

The majority of computer animation made for artistic reasons does not tell stories. 
Films by the Whitneys, Ken Knowlton and Lillian Schwartz, Larry Cuba, Stan 
Vanderbeek and many others are abstract in that their images are non-figurative 
and their dynamics are not dramatic (i.e. are not structured around a story). If they 
are not narrative, then what are they? If they are not structured around a story, 
then how are they structured? Abstract films are often discussed metaphorically as 
"visual music" or "dynamic paintings". An animated film can either be organized 
around stories of "people" interacting with each other and their environments in the 
same way that most literature, drama and entertainment films are or they can be 
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organized internally and rely upon perception and appreciation of their internal 
structure for their value in much the same way that music, dance, or modern 
painting and sculpture do. 

Despite the appearances of the characters, Ani's films are basically dramatic 
or story-oriented and have little in common with abstract animation. Could Ani, 
however, be extended to make abstract non-narrative animation or would a very 
different computer system be required? The changes to Ani would have to be more 
extensive than any discussed so far, but much of the general framework of 
generating and evaluating suggestions from varied sources to determine values of 
the elements of a film should be applicable to this kind of animation. What is 
difficult is deciding how the space of choices or elements should be structured and 
what suggestion sources are appropriate. Also more attention would need to be 
placed upon the global effects and the interactions of the parts to achieve the 
dynamic analog of balance and composition in paintings. It also may turn out that to 
do an adequate job of creating abstract animation good visual feedback is necessary 
and the state of the art in computer vision is not up to the task. 
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Sound or Dialogs 

Anrs films are silent; or if they have sound, it was added by me. Silence is very 
demanding upon an audience and greatly lowers the entertainment value of Ani's 
films. 1 The problem is not one of realizing the sounds— -it would be perfectly 
acceptable if Ani described the sound effects desired and I followed the 
instructions. Rather it is one of describing what sounds are wanted and when. 
Dialog (other than "canned" segments) is beyond the state of the art. Simple sounds 
effects could be added, however. It would be simple to augment the display 
methods to indicate the desired sounds. For example, the "hitting" method could 
specify that a "slugging" sound followed by an "ouch" should happen when the one 
character hits another. This is similar to the way in which "camera" movements 
could be generated. Sound effects could also be suggested by personality 
descriptors or mood changes (if a character becomes unhappy, then a long low 
sound is indicated). Another possible use of sound is for narration. Ani could be 
extended to ask that someone describe what is happening in this segment. The 
description could be the one that Ani uses internally (e.g., (repeat 6 (hits 
stepmother Cinderella))) and the English would be generated by the human narrator 



I. The aesthetic role of sound in film is a controversial subject in film theory. Aesthetics aside, it is 
generally acknowledged that sound is critical in keeping the audience's attention. Good silent films to 
do exist, of course, but they must be very good to also be entertaining. 
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("and then the evil stepmother repeatedly hit poor Cinderella"). 

Section B Making Ani Do Better 

A very different set of improvements and issues arise when the problem of making 
Ani do things better are addressed. Ani can be improved to make better films (but 
not necessarily reasoning any better in the process of making them) or the quality 
of Ani's reasoning could be improved (which hopefully, but not necessarily, will 
increase the quality of the films). 

A Bigger Knowledge Base 

Ani's database is much more meager than it should be. Representing and debugging 
the knowledge in it is a slow process and so Ani knows little more than is needed 
to make the Cinderella film. Too often Ani knows of only one or two ways to do 
something and does not exercise much choice and cannot achieve much variety. 

Were Ani's database to be enlarged to have more alternative methods and 
activities and to have more suggestions from character and relationship descriptors 
then the ad hoc manner in which things are represented might prove cumbersome. 
As more knowledge is added the advantages of a systematic representation scheme 
increase. Duplication of facts or suggestions are unavoidable without a canonical (or 
semi-canonical) representational system. Inference rules, for example, can be 
shared if the actions and objects involved are broken down into more primitive 
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elements. A less ad hoc representation would also alleviate the problem of 
properly weighing a group of suggestions from related sources since the 
relationship between the sources would be more accessible. A representation 
scheme along the lines of Schank's conceptual dependencies, plans, scripts, and 
themes [Schank 1977] could alleviate many of Ani's growing pains. The 
representation would need to be extended to represent descriptors (such as shy, 
dominates, or ugly) in terms of a set of more primitive emotional, inter-personal, 
personality descriptors. For example, the relationship descriptors "likes" and 
"loves" share much in common and breaking them down into more primitive elements 
would enable the knowledge common between them to be placed upon their shared 
primitive parts. 

One might wonder why Ani was not build upon some well thought-out 
systematic representation scheme. It has been recognized in the Al community for a 
long time that a good representation for knowledge is very important. Ani was built 
incrementally as needed. The use of actors (with their local databases and 
procedural attachment) in a specialization-generalization hierarchy in Ani helped 
greatly. One can view Ani's knowledge as pre-compiled in that it consists of large 
interconnected chunks of knowledge instead of smaller parts that could in theory be 
put together to form these chunks. This approach was very practical — it was 
easy to add new pieces of knowledge to Ani and construction was not slowed down 
much by thoughts of how to represent the knowledge in a general and clean manner. 
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The price of such an approach grows as Ani's knowledge grows, but at Ani's current 
size this is negligible and the benefits (primarily ease of implementation) are great. 

A Better Knowledge Base 

The animator must also have an intuitive feel for personality. In our 
business, the animator is the actor, That doesn't mean he has to act 
himself. But he has to give the characters he draws the timing and 
little touches that bring fascinating figures to life on the screen. 

Walt Disney [Thomas 1958] page 134 

Ani's knowledge can be improved by adding more or by making it more systematic, 
^rnm^ but the quality of the knowledge can also be improved. Most of Ani's knowledge is 

possessed by nearly everyone in our culture and is straight-forward to obtain by 
common sense (e.g. that a shy person avoids others, that an evil person hurts 
others, etc.) but some of it is typically possessed only by animators. This is 
especially true of the knowledge inside the definitions of the display methods. Take 
the "standing guard" method as an example. How exactly should the characters 
involved move? Should the guard wait a while (and if so how long) before quickly 
moving to block the prisoner's escape? As the cycle of the prisoner escaping and 
the guard blocking is repeated how should it change? Should the movements speed 
up'.' Should the prisoner almost get away one time only to be forced back? If so 
which time? The ability to answer such questions well is essential for producing 
effective scenes and quality animation. 
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It is possible that Ani is a good framework for placing knowledge and 
procedures together to make animation but that the quality of Ani's animation 
knowledge is poor. The only source of Ani's knowledge is the author who has only 
four years experience making, viewing, and reading about animation and too little of 
that is directly relevant to the problems involved in using movement to convey 
feelings and relationships. The quality of Ani's knowledge could be improved if a 
fast enough computer were available for testing the methods. Just playing back a 
record of the running of a method currently takes too long for anything but a crude 
jerky approximation. 1 Ani's films could be improved by having expert animators 
define the methods and by previewing portions of films to fine-tune the methods. 
The knowledge base could be debugged and fine-tuned still further were Ani to 
*\ make several films that are criticized by knowledgeable audiences. 

Ani was built out of many actors that have very suggestive names such as 
"level of subtlety", "deliberate", "rhythmic", "graceful", and so on. Such names were 
very useful during the implementation since whenever I reached point where such 
factors should be taken into account, I could reference them. The truth is that the 
meaning these words have to Ani is a pale reflection of what they mean to people. 
There are three ways in which this can be improved somewhat. 



/"% 



I. This is on a time shared PDP 10 with a Knight TV Display. Machines fast enough to run Ani's 
films do exist. 
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A Better Knowledge Base Improvements and Extensions 

(a) Add to each one the ability to suggest values and activities (some 
do already). 

(h) Go over all the places where Ani chooses between alternatives and 
make sure that these descriptors are taken into account in a 
reasonable manner. 

(c) Independent of Ani, think about what these words should mean for 
the kinds of films that Ani makes, and then modify Ani, in as 
general a fashion as possible, so that they have these affects on 
the film. 



Attending to Myriad Details 

Joel Moses, leader of the MACSYMA group, once attributed the success of 
MACSYMA 1 to a handful of good design decisions at a very high level, to a large 
number of medium-level decisions, and to thousands of low-level details. His 
point was that for a successful system one needs to pay careful attention to all 
levels and that despite a very good high-level design a system will fail if the 
myriad low level aspects are poorly handled. 



I MACSYMA is a very large and powerful computer system developed at MIT for symbolic 
manipulation of mathematical expressions. 
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Improvements and Extensions Attending to Myriad Details 

Even a charitable view of Ani reveals many low-level deficiencies. If one 
were to glance at the programs that make up Ani one would see many 
comments of the type, "this works ok but should really take into account this or 
that". Here is a sample of real {but relatively minor) deficiencies of Ani. 

(a) Comparison and uniqueness summaries should interact more with 
film descriptors. Finding that the personalities of two characters 
differ suggests that they move differently but this suggestion 
should be tempered by the desired level of obviousness, variety, 
and complexity. 

(b) The conflict resolver should take into account how many of the 
conflicting suggestions were suggested by the sources that already 
had many of its suggestions followed (so to be fair their 

-p-^ suggestions could be ignored in this conflict if need be). 

(c) The movement tendencies such as the tendency to avoid strangers, 
should be used in the prerequisite and "best when" clauses of 
methods. 

(d) Animators often ease a character into or out of a movement. Ani, 
however, has characters accelerate instantaneously despite the fact 
that much effort was expended determining the acceleration of each 
character. The beginning and ends of each movement should be 
influenced by the character's acceleration. This is easy to 
implement if a character is moving towards a stationary target. 
Easing out of a motion (decelerating) is very difficult if its goal is 
moving (since Ani would have to predict when to start slowing a 
character down). 
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(e) Currently the display methods will stretch or shrink an activity's 
running time by changing the speeds of the characters involved, 
changing the number of times a subactivity is repeated, and for one 
method changing the number of degrees a character will turn. If 
the characters involved are not positioned or their current positions 
are so inappropriate that a film cut is required then the choice of 
the initial locations should be influenced by the factor that the 
activity should be stretched or shrunk. 

(f) The degree of deliberateness of the movements of each of the 
characters is determined by Ani. This should influence how often a 
character changes direction, how quickly it does it, and how large 
the change in direction is. 

(g) Each display method is capable of estimating how long it would 
take to run. This estimate does not take into account the influence 
of any gradual changes in the dynamics of a character that is 
happening concurrently. 

(h) The display methods modify the dynamics of the characters, but 
they do so without taking into account many relevant factors such 
as the desired variety level (and what has happened so far), the 
relative speeds, deliberateness, etc. of the characters, and the 
particular locations of the characters at the beginning of the 
activity. 

( i ) When several subscenes need to be temporally ordered, Ani should 
take into consideration where the characters should ideally be 
placed when each one begins and where they will be when they 
end. Ani possesses these facts but does not use them so 
sometimes one subscene leaves the characters in places that the 
following one cannot accept and a film cut occurs while the other 
order would cause no problems. 
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Any one of these problems can fairly easily be rectified. The difficulty is 
that there are very many such minor problems each one of which takes a fair 
amount of programming. Ani is just a prototype system intended to demonstrate 
the feasibility of its design and as such these low-level problems are not very 
relevant. Nonetheless, Ani is partially judged on basis of the quality of the films 
and the reasoning behind them, both of which would be improved by careful 
attention to the thousands of small details such as those just presented. 
Without the attention to such details the reader is forced to accept this thesis 
on the basis of faith, while attention to such details is very time-consuming and 
does not contribute much to the field. The implementation of Ani is built based 
upon a compromise between these two forces. 

Section C How Ani Evolves 

The reader should be skeptical of the generality of the program: is the 
knowledge base "just right" (i.e., finely tuned to elicit this one chain of 
behavior)? The answer is 'No". (The design of AM was finely tuned 
so that the answer to this question would be "No". Ponder that one!) 
The whole point of the project is to show that a relatively small set of 
general heuristics can guide a nontrivial discovery process. Each 
activity, each task was proposed by some heuristic rule (like "look for 
extreme cases of X") which was used time and time again, in many 
situations. It was not considered fair to insert heuristic guidance which 
could only "guide" in a single situation. 

-Douglas Lenat [Lenat 1976] p. IS 
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When Ani made sections of film that I did not like, I found out why and looked 
for faulty reasoning. If it could have been made it better if Ani had taken some 
missing factors into account or had available more accurate information then I 
would change the program. For example, Ani did not originally make the cut in 
the fourth scene just before Cinderella and the Prince move to each other. The 
routine that approved the locations of the characters was not rejecting their 
locations because they were in contact. Clearly the method for displaying that 
two characters move to each other should know that if they are initially too 
close together then the characters should be relocated somehow so that they 
can be seen moving towards each other. The method was relatively easy to 
modify this way but the bug persisted. It turned out that Ani had lost track of 
the locations of the characters because many of their previous movements were 
to moving targets and some of the characters were stopped before finishing. 
Aside from completely simulating the characters involved,' I see no way that Ani 
could have figured out their locations at the beginning of this subscene. The 
previous activity was one which left Cinderella and the Prince in contact and so, 
even though Ani could not figure out where they were, that they were close 
together should have been obvious. Ani had to be extended slightly so that 
activities could record the relative positions of its participants after they run 



I. Detailed simulation is typically much more expensive than prediction or approximation and 
accuracy is not needed. Also it seems very unlikely that humans animate by detailed simulation. 
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and so that this record was accessible to the process that determines the 
relative distance between two locations. It was easy then to add to the 
previous activity, "accompanies each other", the fact that it left its participants 
very close to each other. 

The only legitimate changes that can be made so that Ani makes choices 
more to my liking are to add obvious information that was lacking or was too 
inaccurate (this happened often in decisions based upon very crude estimates of 
the time required to run an activity). The only other change that is legitimate 
to make to Ani is to take factors into consideration that obviously should have 
been but weren't. 1 

Section D Critics and Criticism 

The process just described is a good one for constructing Ani but is not a 
reasonable method for a user trying to control Ani's decisions. Instead Ani 
should be extended to accept criticism of the films and their justification. For 
many types of criticism there is an obvious response. If one says that a 
particular activity in a subscene was poorly chosen then Ani could redo that 



6 



I. Other changes would constitute cheating in that I would be arranging for Ani to do what I want 
without improving the reasoning. For example, if Ani chose poorly between three choices I could have 
(but did not) fix things so that the option no longer existed or looked undesirable. 
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Critics and Criticism Improvements and Extensions 

subscene selecting any reasonable alternatives to that activity. 1 If one says that 
a section of the film happened too slowly or quickly then the display methods 
involved could be asked to replan the activities involved shrunk or stretched 
respectively. If one says that during a particular scene two characters should 
not get very close to each other (as one might be tempted to say about 
Cinderella and the Prince in the first scene) then this post-condition could be 
added to all the methods in that scene and it could be remade. If one looks at 
Ani's reasoning for a subscene and sees that Ani failed to make a cut because 
an estimate of the distance between two characters was not accurate enough, 
then one could ask Ani to remake the subscene as if the distance were known 
to be some particular value. 2 

If Ani had such a mechanism for accepting user criticism then it could also 
be used for responding to "self criticism". This criticism could be generated 
straight-forwardly in many instances. For example, if Ani made a decision based 
upon a prediction of the distances between two characters and then when finally 
running the film found out that the prediction was way off, then Ani could 
remake the section involved with an accurate figure for the distance between 
the two. Or if Ani estimated that a subscene would take a certain length of 



1. This could easily be implemented by having Ani check a record of user criticism whenever faced 
with several alternatives and removing those alternatives that were vetoed by the user. 

2. Notice that most of the changes could be made locally since Ani knows what other parts depend 
upon the part being changed and could decide if the change should propagate to other parts. 
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time and then when running it found out that the estimate was way off, the 
section couid be remade with an accurate estimate. This ability would have 
improved somewhat the fourth subscene, where too many things happen too 
quickly and part of the problem was that Ani's estimates for the running time 
were too large. The ability to remake a choice after being told that at least 
one of the alternatives is no good could also be used internally by Ani. If Ani 
makes a choice that leads to various problems (such as failure to meet the 
prerequisites of any the suggestions available for carrying out the choice, or if 
the same choice was made later on without any alternatives and the desired 
variety level of the film is high) then, if possible, the affected parts could be 
remade avoiding the problem. The usefulness of this kind of self criticism would 
depend upon an ability to remake only those sections involved (rather than 
remaking the film from scratch) which in turn depends upon Ani keeping good 
records of why decisions were made the way they were and what else they 
depended upon. 

The approach just outlined seems promising for handling (and generating) 
criticism of many types. Much criticism does not fit into this mold. Ani 
currently has practically no conception of rhythm, for example, and so could not 
respond to criticisms of the film's rhythm. The same holds for notions such as 
climax, tension, mood, or humor. Any aspect of Ani that is built-in (as opposed 
to explicitly taken into account) or lacking entirely cannot be criticized with any 
effect. Many open research questions lie in the making of this more general 



Page 191 



xr 



^"S 



Critics and Criticism 



Improvements and Extensions 



criticism-taking ability. 



/""%, 



Section E Self-Generated Film Descriptions 

A very intriguing direction for future research is to consider how a system might 
generate its own film descriptions. 1 Not oniy must good descriptions of the 
characters, their relationships, the scenes, and the stylistic parameters of the 
film be made, but these different components of the film description must fit 
together well. Clearly any one of these problems in generating film descriptions 
is a large research project. 

Describing the Characters 

The main problem in generating descriptions of the characters is that one needs 
some theory of what makes a good description of a character, of what makes a 
good set of characters, of how the characters should be related, and so forth. 
The problem is complicated by the fact that modern literature abounds with 
intentionally self-contradictory description of characters. The essence of many a 
novel is the interplay of conflicting forces or aspects of a main character. 



I. Making film descriptions of the quality and complexity of real films entails so many different 
aspects of intelligence and such large bodies of knowledge that it will be a long time before anvone 
other than humans will do this. 
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I would suggest to anyone trying to pursue this that they begin with a 
scheme in which there are several prototype characters (e.g. villain, weak 
mild-manner good guy, etc.). A character prototype is selected, instantiated, and 
then modified to produce a character description. 

Describing the Scenes 

Meehan's Talespin system [Meehan 1976] provides some hope that scene 
descriptions could be automatically generated without making the system 
completely artificially intelligent. Talespin is given character descriptions and 
generates plausible activities for them. The difficulty is that plausible activities 
are not enough, they need to be interesting, to fit into some larger scheme and 
so on. 

One avenue of exploration begins with a source of stories (say, dreams or 
fears) from which are constructed general templates for films. A combination of 
a very few of them could then be the starting point of a film. 
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Describing the Style and Length 

Again, a beautiful object, whether it be a living organism or any whole 
composed of parts, must not only have an orderly arrangement of 
parts, but must also be of a certain magnitude; for beauty depends on 
magnitude and order... As, therefore, in the case of animate bodies and 
organisms, a certain magnitude is necessary, and a magnitude which 
may easily be embraced in one view; so in the plot, a certain length is 
necessary and length which can be easily embraced by the memory. 

- Aristotle (Aristotle] p. 653 

How does one decide that one film should only be a few minutes long, 
while another should be half an hour long? How does one determine that one 
/""""% film should be varied, complex, subtle, and low energy while another should be 

simple, obvious, flashy and slightly incoherent? Perhaps these things are 
suggested by the purpose of the film, by a model of the expected audience, by 
the point of the film, and other high-level aspects of the film. 

Section F Incorporating a General Theory of Aesthetics 

There is no "aesthetics expert" that I can point to in the implementation of Ani. 
There are no procedures within Ani for deciding that one alternative will lead to 
a more aesthetic product than the others. One cannot pull out a part of Ani 
and use it to make beautiful music or paintings. 
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Is such an aesthetics expert theoretically possible or is aesthetics the sort 
of thing that is not isolable, but is instead distributed throughout thousands of 
different experts and heuristics for making decisions? Many Al researchers 
doubt that a really "general problem solver" is possible, that instead the 
problem solving ability must be distributed and specialized. Is aesthetics similar 
to problem solving in this respect? 

Ani's aesthetic expertise (what little there is) is spread out in the 
procedures that resolve suggestion conflicts, that decide what choice point to 
activate next, that try to satisfy the global film descriptors, that try to 
determine the dynamics of the characters in accordance with both the 
^.^^^ descriptions of the characters and their relationships and so on. Could all these 

procedures instead call upon a "general aesthetics problem solver" for advice? 

There have been a few attempts to formalize a general procedure that 
could measure the aesthetics of an object. Two contemplated structures could 
be measured and the one with more "aesthetic value" could be chosen. Section 
B "Aesthetic Systems" on page 133 describes such attempts and their 
shortcomings. 

During the construction of Ani I was skeptical that such an "aesthetics 
expert" could be constructed for the kind of animation that Ani makes, much less 
a general expert that would be applicable to music, poetry, sculpture, etc. 
There was no attempt, therefore, to build Ani with one. Recently I have 



/*"> 



Page - 195 






f*\ 



/"""x 



o 



Incorporating a General Theory of Aesthetics Improvements and Extensions 

become more optimistic and would recommend this as a very interesting line of 
further research. I am beginning to think that there is something in common 
between all aesthetic objects. The aesthetics of an object lies in the structure 
of its description and how that description relates to other structures in the 
artist's or viewer's mind. The way that the different parts, levels of 
abstraction, aspects, and views of the object relate to each other, the whole, 
and related structures is crucial. The pattern of the justifications of the design 
of the parts of the object also plays an important role. 

What I am a suggesting is that perhaps a generalization of the formalist 
school of aesthetics might lead to an understanding of aesthetics appropriate for 
building a computational aesthetics module. 1 One need not wait for a better 
understanding of art to come from aestheticians, psychologists, or art historians 
before attempting such an ambitious project because they iack the concepts 
from computer science and Al of processes, recursion, hierarchies (and 
heterarchies), and representation. This is basically the same reason that is 
commonly put forth as to why Al researchers have something to say about 
learning, meaning, causality, language and the like that educators, psychologists, 
philosophers, and linguists have not been saying. This is not to say that only Al 
has much of importance to say, to the contrary, only that a computational 



I. Over fifty years ago. Roger Fry exponded the "formalist school of aesthetics" which stresses the 
form of an art object, i.e. how the different perceptual aspects of the object inter relate [Fry 1974]. 
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viewpoint has its own unique contributions to make. 

Section G Making Ani Smarter 

Ani could do a better job with a better database, and an even better job if 
"smarter", if Ani couid take criticism into account, maybe even self-generate 
some, the films would be better (as described above). If Ani took into account 
more the interdependencies of the suggestions in making choices, the films would 
be better. If Ani could let the methods interact more with each other and with 
the determination of the dynamics of the characters, the films would be better. 
If Ani knew something about how to create tensions and expectations in viewers 
gr\ or knew more about themes, morals, points, and focuses of a story or scene, the 

films would be better. These are all very worthwhile extensions to Ani that 
are necessary if Ani's films are ever to be close to human quality and variety. 

There is at least one other way that the basic top level strategy of Ani's 
filmmaking could have been organized. Rather than Ani's current tendency to 
construct character and scene descriptions from many divergent sources as 
needed, a knowledge-oriented animator could be built whose basic operation is 
to match the user's film description to some prototype film and then modify that 
prototype to fulfill the film description. This is based upon a well-known 
technique used in Al problem-solving and automatic programming systems (see 
[Sussman 1977], for example). A proto-film matching system is in the spirit of 
frame systems [Minsky 1975] or scripts [Schank 1977]. It is a rare and difficult 
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event in one's mind when something complicated is constructed de nova, the vast 
majority of one's thoughts use already constructed "stereotypes" which are 
modified to fit the current situation. 

One might object that even though this would be a successful method for 
film production, it would not be creative — the machine would only be spitting 
back films that were put into it. This would depend upon the number of 
proto-films that it selected among, how intelligent that selection was, and how it 
modified the prototype to fit the situation. One can concentrate upon being 
creative in the choices involved in the myriad low-level details of an aesthetic 
object (much like a medieval artist painting a crucifixion where the high-level 
structure is given and the art lies at a different level). Or one can concentrate 
upon the high-level structure and content and pay less attention to details (the 
extreme version of this is "conceptual art"). Of course, there is a danger that 
the films of the proto-film matching system will be the result of applying a 
"once successful" formula one time too many. 1 The two approaches are not 
incompatible, indeed one would expect an ideal version of Ani to be able to do 
both and intelligently choose between them for each episode. 



I. According to some this is exactly the trap that the Disney studio fell into in the late fifties and the 

sixties [Schickel 196$]. 
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Section H Where All This Might Ultimately Lead 

It is quite clear in what direction man's symbiotic relation to the 
computer is headed: if the first computer was an abacus, the ultimate 
computer will be the sublime -aesthetic device: a parapyschological 
instrument for the direct projection of thoughts and emotions. 

Gene Youngblood [Youngblood 1970] page 139 



Imagine a James Bond movie in which you can variously be the hero, 
villain, and bystander, whose plot is fluidly controlled by decisions you 
make in the course of its action, all presented in real-time full color. 
Your family and friends can assume roles also — the outcomes will be 
the result of all your decisions. All of this and all of the variations of 
,/**" N this theme that you can imagine will be possible (and inevitable) just a 

few years from now. 

Alan Kay, A dele Goldberg, and Larry Tesler [Kay 1978] page 30 
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Imagine a machine of the future that is the ultimate entertainer. You 
just sit down in your living room, tell it that you would like to see 
"Gulliver's Travels", and an image on your wall begins to form. You 
see, in three-dimensional color, a person looking the same as you 
wearing 19th century English clothes, standing on the bridge of a 
sailing ship of the same period. You hear the waves in front of you 
and the cry of a seagull behind you. You breath in the fresh salty air 
and say "No I think I'm in the mood for some drawn animation, 
something like 'Snow White' maybe" and the image blurs and refocuses. 
You see a painting of a man on a old sailing ship. Large waves are 
hitting the side of the ship and dark clouds move swiftly. The air is 
heavy and moist... 

Ken Kahn (from an unpublished science fiction story) 



f\ Ani's film-making ability is very limited, but need it be? Are there any 

theoretical limitations involved or could a computer system like Ani, in principle, 
make any desired film? I personally think there are no theoretical limitations but 
that great progress on Al problems of representation of knowledge, common 
sense reasoning, and learning is required to create an "ultimate" film maker. 
This is not to say that Ani could not be improved significantly without great 
progress in Al. On the contrary, the extensions to Ani described in Chapter X 
"Improvements and Extensions" beginning on page 162 point in the direction of 
the eventual removal of many of Ani's limitations. 
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Towards what sort of system is this research simply a very small first 
step 9 It would be the product of substantial progress in Al, computer 
technology, and much more computational research on aesthetics and creativity. I 
foresee no theoretical obstacles to this system ultimately being able to 
automatically produce any kind of film or tv program that is now produced. In 
other words, it could create movies ranging from those with real looking people 
doing everyday things (like Star Trek) to ones containing only abstract dynamic 
images (like parts of Fantasia). 

One purpose this fantasy is to give direction and inspiration for the building 
of extremely crude approximations to this ultimate 

/""> entertainer/envisioner/fantasier. Another is to give the reader a perspective of . 

where this research might fit into a bigger picture. It describes where I see 
this kind of application of Al to animation could eventually lead. 

Many computer graphics researchers are involved in taking steps from a 
different direction towards this same ultimate system. They are discovering 
techniques for producing realistic images from an imaginary camera filming an 
imaginary world and are now able to make very realistic teapots and fairly 
realistic human faces ([Parke 1972] and [Blinn 1976]). Others are working on 
three-dimensional display systems, three-dimensional sound synthesis, and many 
other components of this ultimate system. What appears to be lacking in their 
vision of the "ultimate" system is an appreciation for the role that Al must play 
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Where All This Might Ultimately Lead Improvements and Extensions 

in such a system. This thesis concentrates not on the technical issues of 
generating high quality images, sounds, or smells given a description of an 
imaginary world, but rather on the creation of descriptions of these imaginary 
worlds and their laws and events. 

The movies of this ultimate system can differ from the cinema of today by 
interacting with the viewer and by automatically customizing itself to the 
viewer's idiosyncrasies. At any point in the viewing of a movie, one can change 
the course of the plot, alter a character or change the style or mood, or 
actively participate in the events of the film. Also, this Ani of the twenty-first 
century can mold the film to what it knows of the tastes, preferences, and 
idiosyncrasies of the viewer. (See [Negroponte 1976] for a discussion of 
idiosyncratic systems.) 

After imagining the impact of this ultimate system on entertainment, it takes 
little additional imagination to see what the impact of it on education could be... 
Or communications... Or art... 

Don't misunderstand this. People will probably still make movies because 
they want to, its just that everyone will be able see movies custom-made to 
their interests, preferences, and moods while actively participating in their 
creation. 
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Improvements and Extensions Where AH This Might Ultimately Lead 

Many have moral objections to this dream. They fear that people will 
escape reality into the dreams constructed by this dream machine. Maybe. On 
the other hand, the active participation and control possible make it clearly 
superior to the contemporary problem of people "escaping" by passively 
watching commercial television. 

In any case, it will be a long time before this ultimate system exists. 



By embodying all the deterministic knowledge in the machine (such as 
of the English Language, of the rules of logic, of harmony and melody) 
and by building in an enormous store of previous experience which 
allows preselecting elements likely to succeed, the probability space 
can be enormously restricted. Moreover, the machine can be its own 
critic and censor. I believe, though, that such a machine will be hardly 
less complicated than the human brain, and therefore there is some 
hope that it will never be built. On the other hand I should welcome 
simpler machines, such as Orwell's "versifier" which produces popular 
lyrics "untouched by human brains" for debunking all that is mechanical 
and bogus in what passes by the name of art. 



Dennis Carbor [Reichardt 1971] Page 18 
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Discussion 

Chapter XI Discussion 

This chapter is intended to puli things together and discuss this research from the 
perspectives of what it means for the understanding of creativity, aesthetics, and 
style, it also discusses some of the objections to the relevance of this research to 
questions in art theory. Finally it attempts to review the many different viewpoints 
of Ani that have been presented in this thesis. 

Section A Ani and Creativity 

The fanatics, finally, accept without hesitation all the marvels of the 
^^^ craziest science fiction. The moon? Why not— it's quite within our 

f^- reach. Longevity, too, is just around the corner... Why not the 

creative machine as well? 

- Iannis Xenakis in 1963 [Reichardt 1971) page 124 



In a sense, the computer with its program could be considered 
creative, although it can be argued that human creativity was involved 
in the original program with the computer performing only as arr 
obedient tool. 

- A. Michael Noll [Noll 1967] page 146 
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Discussion Ani and Creativity 

The creativity is in the choice of the algorithm, not in the grinding out 
of the variations, which represent the micro-aesthetics of a picture. 

Leslie Mezei [Reichardt 1971] page 165 



Relying Upon Knowledge Rather than Simulation 

Ani's ability to create is due to the use of knowledge in making choices. Ani makes 
a choice by knowing enough to make reasonable decisions. An alternative method 
of making choices is simulation. For example, rather than decide what should 
happen in the second scene in order to convey that the stepmother prevents 
Cinderella from meeting the Prince, we could simulate each of the characters and let 
y-s^ thern "decide" what happens. This is essentially the approach taken by Meehan in 

his Talespin system [Meehan 1976]. Talespin simulated characters and then told 
stories based upon the simulations. This may produce more realistic stories but 
they will typically be less artistic. 1 A writer when faced with a choice of actions 
for a character does not base his or her decision solely on what the character is 
mostly likely to do in that situation. Instead higher-level purposes (such as the 
desire to create an interesting story) are primary and so the characters are 
constrained only to act in a possible manner (but not necessarily the most 



1. It should be mentioned that Talespin was not intended to produce art but instead was a study of 
plans and high levei symbolic simulation. 
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probable). 

Ani does eventually simulate the characters, when finally running the actions. 
But by that time the characters are so restricted in what they should do that they 
behave as desired. This is partly due to the fact that Ani makes plans and then 
arranges the positions and behavior of the characters to fulfill some high-level goal. 
Simulation has its place but knowledgable choice and planning are needed to create 
aesthetic (as opposed to realistic) objects. 

Who Made the Cinderella Film? 



/"■\ 



This brings us back to the issue of the "who" who composes computer 
music. In most circumstances, the driving force behind such pieces is a 
human intellect, and the computer has been employed, with more or 
less ingenuity, as a tool for realizing an idea devised by the human. 
The program which carries this out is not anything which we can 
identify with. It is a simple and single-minded piece of software with 
no flexibility, no perspective on what it is doing, and no sense of self. 
If and when, however, people develop programs which have those 
attributes, and pieces of music start issuing forth from then, then I 
suggest that will be the appropriate time to start splitting up one's 
admiration: some to the programmer for creating such an amazing 
program, and some to the program itself for its sense of music. 

-Douglas Hofstadter [Hofstadter 1979] page 60S 



I. Talespin could set the initial conditions so that its story would have a particular moral and in this 

regard is similar to Ani. 
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Discission Who Made the Cinderella Film? 

When Ani makes a movie who should be considered its author, Ani or Ani's 
author 7 Ani's maker is the author if either the movie is described with so much 
detail that Ani has few choices of any consequence to make or if the movie 
implicitly existed in Ani before. The first case is the most common case for 
computer animation — the programmer is the author. The latter case hinges 
strongly upon what it means for something to implicitly pre-exist in a computer 
program. One needs to judge not only the product but also the processes that 
made it. If the processes are superficial or syntactic then the product pre-existed 
even if it was transformed and its parts combined. 

Ani made the Cinderella film after being presented with a very incomplete 
^ m * s ' description and constructed it in a fairly general fashion out of many small parts 

which were chosen by using complex rules and several different bodies of 
knowledge. This raises the question of who really made the Cinderella film — Ani 
or me'' Ascribing authorship to the final product in this case is a matter of degree. 
The level (and number of levels) of reasoning is an important factor. So is the 
number of choices available and the amount of knowledge brought to bear in a 
sensible manner in the creation. The generality of the creation process is also 
relevant — how many different films can be made and how different could they 
be? Ani's reasoning is pretty crude and simplistic (in comparison to humans). 
Nonetheless, Ani is further in the direction of being an artist than other computer 
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Who Made the Cinderella Film? Discussion 

programs for producing aesthetic objects. 1 

implications of Accepting Ani as Creative 

Stated otherwise, we would have a satisfactory theory of creative 
thought if we could design and build some mechanisms that could think 
creatively (exhibit behavior just like that of a human carrying on 
creative activity), and if we could state the general principles on which 
the mechanisms were built and operated. 

Newell. A.. Shaw. J., and Simon. H. [Newell 1962] page 64 

If one accepts the idea that Ani is really the one who created the Cinderella 
/*"\ film, (or at least is a creative collaborator) then the processes that went on within 

Ani during the creation take on a new significance. The popular view of creativity 
is that it is a mysterious, subconscious, intuitive process. Creative people are 
thought of as "black boxes" that produce by unknown processes (unknowable 
according to some). Ani is a creative entity that is a "glass box" that we can look 
inside and at any level of detail see what is happening. As a tool for understanding 
creativity, Ani is even better than a glass box because one can perform 
experiments. We can not only look at what Ani is doing but can also change the 
system and observe the effects. For instance, one can study the relevance of a 
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l. An essav on ai and art can be found in Appendix VIII, "An Essay on Art and Am" on page 291. 
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Discussion Implications of Accepting Ani as Creative 

particular body of knowledge by changing or disenabling various portions of that 
knowledge and having Ani perform the same task before and after. 1 

Objections to the Notion that Ani is Creative 

One objection to this use of Ani, is that what one learns about will be machine 
creativity and not human creativity. A weak response to this objection is to agree 
but claim that at least we have discovered that not all forms of creativity need be 
mysterious. At least one form of creativity has been explicated and perhaps is 
even teachable to humans. The stronger response is to say that there are no 
significant differences between human creativity and Ani's. At first glance this 
^" v * seems wrong; many significant differences come to mind. Ani only does what it's 

told, humans only sometimes do. Ani makes movies by following rules; people are 
most creative when they break the rules. Ani can't be genuinely original or 
surprising; people can. Ani can only create a very limited class of animated films, 
while creative humans possess much more general capabilities. Ani's films are not 
as good as those made by professional animators. Ani is too simple-minded and so 
on. Let's consider these objections one by one. 



/ I **** v , 



I. Such experiments were not performed mostly because of large amount of time involved, since each 
run of Ani is very time consuming. Read Appendix III. "Time and Memory Statistics" on page 259 to 
see exactly how much time and computer resources are involved. 
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Ani Can Only Follow Rules Discussion 

Ani Can Only Follow Rules 

The first two objections are essentially the same. Ani follow rules — both internal 
and external ones — while people can break out of any rule. Both 
characterizations are incorrect. Ani can break rules in a real sense and people (and 
Ani) in another sense cannot. The different senses concern different levels of rules. 
Ani has many rules that determine whether a rule (or what I prefer to call a 
suggestion since they are broken so often) should be followed, modified or rejected. 
For example, in selecting a default acceleration for the Prince, the rule that evil 
people tend to have high accelerations was used to generate a suggestion that the 
Prince's acceleration be low since he is described as good and good is the opposite 
of evil. But this suggestion was found to be in conflict with the suggestions from 
"strong", from the comparison of the Prince and Cinderella, and from some of the 
global descriptors of the film and so was thrown out. There are even rules for 
deciding the precedence of the rules for deciding which rules to follow and which to 
reject. But there are always a few rules, possibly only one, at the top of this 
structure which are not supposed to be broken. In Ani, for example, there is a rule 
or procedure that ranks the conflict resolving rules and determines which rules will 
be tried first. 
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DisctiMion Ani Can Only Follow Rules 

When these top-level rules are broken there must be bugs in the 
substructure that supports the rules. The substructure itself is a computer 
program which strictly follows the rules of which it is composed and bugs prevent it 
from implementing the top-level rules as intended I know of no reason to think this 
situation does not apply equally well to humans. People have many levels of rules 
and when one says an artist has broken some rules (either internal or external 
rules) there usually exists some other rules (perhaps unconscious ones) that the 
artist applied in rejecting the broken rules. Either some "meta-rules" were invoked 
or else some bugs occurred during the execution of the rules resulting in 
"unintended" behavior. The latter case is often labeled insanity when it occurs in 
^*n, people. Indeed there may be some truth to the belief that some of the very 

creative people are insane or at least mentally unstable (e.g. Van Gogh, Darwin). 
This does not mean that most crazy people will be creative since there is much 
more to creativity than the unintended breaking of rules. 
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Ani Can Only Follow Rules Discussion 

Creativity and Randomness 

A brief word about random choices is called for. When a choice has to 
be made and there's no theory around to make it, TALE-SPIN chooses 
at random Choosing the first name of a character is an example. The 
storyteller has no reason to prefer "John" over "Arthur" or any other 
name in the list. There's nothing which the storyteller has in mind 
which fixes the choices of the first name, so it chooses at random. If 
Arthur Bear is hungry and doesn't know where any food is, he'll choose 
a food at random and start looking for it. Obviously, that's a very 
simple strategy. A "theory" about choosing a food would require a 
goal calculus -- what food is nearby? Cheap? Available? But the point 
is that not even random choices are free of side-effects. It makes a 
difference which food Arthur Bear picks, since he'll look in different 
places for different foods. 

James Meehan [Meehan 1976] page 161 



In many areas of art and music it has been commonplace for the artist 
to tell you there's nothing in his work that doesn't have some sort of 
valid relationship or meaningful reason for being there. They've 
constantly sought to avoid arbitrariness— not accident: you can often 
make an accident turn into a very wonderful twist to new meaning. 
But the worst kind of arbitrariness is when a person thinks his own 
casual decisions are great simply because he's done it, because he 
decided to be arbitrary. 

- John Whitney. Sr. [Youngblood 1970] Page 221 
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Discussion Creativity and Randomness 

They [scientists] think if it's art, it's free. Anything that goes with 
random numbers is art; and anything that has to be worked out 
carefully so that this goes here and this has got to go there, that's not 
art, that's science. But for my money it's more important and difficult 
to get this here and that there in the area of art, because it involves 
much more than just counting numbers and making it mathematically 
sound: it's got to be intensely and intuitively sound. That's what I'm 
searching for. That's what I mean by structure. 

John Whitney, Sr. [Youngblood 1970] Page 222 



// is a common notion that randomness is an indispensable ingredient of 
creative acts. This may be true, but it does not have any bearing on 
the mechanizability — or rather, programmability! — of creativity. The 
world is a giant heap of randomness; when you mirror some of it inside 
f^ your head, your head's interior absorbs a little of that randomness. 

The triggering patterns of symbols, therefore, can lead you down the 
most random-seeming paths, simply because they came from your 
interactions with a crazy, random world. So it can be with a computer 
program, too. Randomness is an intrinsic feature of thought, not 
something which has to be "artificially inseminated", whether through 
dice, decaying nuclei, random number tables, or what-have-you. It is 
an insult to human creativity to imply that it relies on such arbitrary 
sources. 

- Douglas Hofstadter [Hofstader 1979] page 673 

The idea of a machine that could create objects of art is old. Mechanical devices 
for generating prose go back several hundreds of years, Jonathan Swift, for 
example, in "A Voyage to Laputa" writes satirically of a machine with words 
attached to gears that could generate sentences by successively choosing a word 
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randomly from a set of words associated with each position in the sentence [Swift 
1 726]. With the advent of computers many people have created systems that they 
claim have created drawings, animation, music, and poetry. To my knowledge, nearly 
all such systems fall into two classes — either systems that are best thought of as 
an artist's medium, i.e., a collection of tools for the human artist to use or else 
systems which make choices between alternatives on an arbitrary basis, usually by 
using a "pseudo-random number generator". With the former the creative impulse 
is entirely from the human user, while the latter is essentially not much different 
from flipping coins to create art. 1 

While the purposes of such systems overlap with those of Ani, my approach 
is at an opposite extreme. One of Ani's design criteria is to minimize the 
arbitrariness of any choice Ani makes. The idea is that for every choice Ani should 
have available large bodies of knowledge, gather together that which is appropriate, 
and make a reasonable choice. My view of aesthetics gives much importance to the 
quality of the reasoning behind a work of art. The art in an art object lies at least 
as much with the perceived quality of the thinking behind each choice as with the 
perception of the object itself. 



I. In domains requiring' much creativity and aesthetic judgment there have been a few systems that, 
like Am. fit in neither of these classes. Meehan's Talespin uses knowledge and symbolic simulation to 
make storips and Lenat's "artificial mathematician" makes (aesthetic) judgments about the quality of 
newly created mathematical concepts. These systems are discussed in Section F "Other Systems that 
Create Aesthetic Objects" on page 47. 
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I am not saying that randomness and arbitrariness do not have their place in 
art. Their place is where an artist uses them deliberately for good reason. A 
slightly perturbed square, for example, may in many cases be more desirable than a 
perfectly drawn square, maybe to look more like a square drawn by a human in a 
hurry. I am aware of no computer program, however, that decides to use 
randomness. That has always been the decision of the programmer. [Reichardt 
1971] 

Good art is carefully constructed and thought out, each element is there for 
good reasons. A random number generator is as likely to produce a "Mona Lisa" as 
are a million monkeys to type "Hamlet". This is not to say that there are not 
inconsequential decisions that have to be made that could just as well be made by 
the toss of a coin. In such cases, however, one can just as easily ignore the random 
number generator and produce just as good a film. 

Some have speculated that creativity is essentially a controlled use of 
randomness. A common idea is that large correlation matrices be used to restrict 
the next thing chosen to the set of values that have ever occurred immediately 
after the previous n values. 1 If the values are letters, n is 3 or 4, and the 
correlation matrix is based upon text of the works of particular authors then the 



I. Sf" [Bennett 1977] for a presentation of this idea. 
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Creativity and Randomness Discussion 

output "bears a striking resemblance to the authors' styles" ([Bennett 1 977] page 
702). The problem is that as the order of the matrix increases then the output 
becomes less and less original since it is more restricted to doing exactly what was 
done before. As the order of the matrix decreases the originality increases but the 
chance of finding anything interesting, correct, beautiful, or whatever becomes 
miniscule. Not only does randomness rarely produce art, but its use just shifts the 
problem of creativity to the equally hard problem of selecting the good stuff among 
the astronomically large number of products of a random process. 1 

Can't Be Original or Surprising 

r^ On one occasion I met Picasso when I had just left the subway. "I have 

seen workers removing old posters from the subway and pasting up 
others," I said. "Some of the surfaces are covered with fragments of 
previous posters and the effects are beautiful." "Yes," he answered, 
"nothing is an accident. A man destroys here, puts something there. 
There is something mysteriously conscious and deliberate that takes 
place in the mind of the man who pastes and tears those posters. The 
result is not only accident. " 

Xavier Gonzalez [Ashton 1977] page 91 

That Ani cannot really be original or surprising is another objection to the idea that 



i. Jorge Borges in his short story "Library of Babel" [Borges 1962] conveys very well the uselessness 
of a library th.it contains every possible book (less than 410 pages) despite the fact that it contains 
many undiscovered masterpieces and secrets of the universe. 
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Discussion Cant Be Original or Surprising 

Ani is a good model of creativity. 1 Ani can only do as programmed and the 
originality or ability to surprise one might perceive is just an illusion, since it was 
all there before. But what does it mean to say it was all there before? If all that 
was there were complex rules for combining many disparate chunks of knowledge in 
accordance with the film description then the film was not there before. Similarly 
the objection that Ani could not genuinely surprise anyone who understood the 
program loses its force when the complexity of the rules and amount of knowledge 
involved in making a film are great. I am often surprised by what Ani does. 2 

There is another sense in which Ani really can be original or surprising. Ani 
does have a very crude "understanding" of originality which translates to rules such 
as avoid making the same choices as previously and avoid choices that are known to 
be stereotypical. Ani does not currently have any notion of surprise but one could 
be constructed that deals with the illicitation and subsequent breaking of the 
expectations of the viewer. In other words originality (and the ability to surprise) 
are manifestations of the actions of a few rules upon large bodies of knowledge. 



1. This is old objection that is often attributed (perhaps wrongly) to Lady Lovelace in her writings on 
Babble's Analytic Engine [Taylor 1843). This issue is discussed at some length in [Turing 1950). 

2. I was surprised when Ani ended the first scene by having Cinderella meet the Prince and 
accompany him across the screen. Upon reflection 1 realized this was not a bug. It is perfectly 
consistent with the story as described to Ani. 
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Ani is too Limited -- too Simple-Minded 

It is easier to and you will learn a lot more from studying somebody 
who is a very bad amateur composer than from studying a good one 
because the good one presumably has 20-50,000 subroutines of 
various well-developed sorts whereas the other one has only a few 
hundred and you might gain some understanding by that. The worst 
thing you could possibly do is to try to figure out the processes of 
very exceptional composers who have been dead a long time. 

Marvin Minsky [Chaikin 1979] 

Another potentially significant difference between Ani's creativity and a human's is 
that The variety of creative acts Ani can perform is very limited. This is of course 
to be expected since Ani's knowledge is restricted to that which is useful in making 
a very simple kind of non-figurative narrative computer animation. This difference 
between Ani and people is not qualitative but is in the amount of knowledge 
available. 1 It is possible that people are creative in virtue of the large number of 
different bodies of knowledge they possess. Ani cannot make analogies between 
fields, cannot cross-fertilize different areas, so perhaps is a bad model of creativity 
after all. It may turn out to be the case that this cross-fertilization is just a special 
case of creativity that itself is best viewed as an ordinary creative problem (i.e. 
how to find two or more fields and apply knowledge in one to accomplish something 



i. Very large quantitative differences are qualitative differences though. 
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in the other). 

Ani may be too simple minded to be a good model of creativity. Ani's 
reasoning is too crude and knowledge too limited. In the system's defense, Ani may 
not be a very good artist, but being relatively simple is a more comprehensible 
object of study. We should start with simple models, understand them, and then 
build performance-oriented ones. One calls a large complex computer system 
simple-minded only with respect to humans performing the same task. Ani is a very 
complex model in comparison to most psychological models — complex enough to 
actually make movies. 

sr\ Ani's Films Are Not Good Enough 

Some of the programs we have described perform work that is 
considered difficult, and even mildly creative, when it is done by 
humans. Although these programs fall considerably short in 
performance of the highest levels of creativity of which humans are 
capable, there is every reason to suppose that they are qualitatively 
of the same genus as these more complex human problem solving 
processes, 

Newell, A.. Shaw, J., and Simon. H. [Newell 1962] page 1 16 
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Yet another objection to Ani as a model of creativity is that Ani's films are 
not as good as films made by humans. A model of the process of creating mediocre 
art objects is not too interesting according to this view. Again the issue is whether 
the difference in quality is due to fundamentally different processes going on in 
humans and Ani or if it is just a matter of degree. Seymour Papert's super-hurnan 
human fallacy is apropos here. Why should Ani be compared to the best human 
animators 9 Why not compare Ant to a ten-year old faced with the same task? 
Some may be willing to grant that Ani is a model of mediocre creativity, but that 
that is not the same activity that occurs when great artists or inventors create. 
Personally, I believe the difference between the very creative and the non-creative 
j0»\ is due to much larger, better organized bodies of knowledge and better rules for 

using that knowledge appropriately. Even if it turns out that there are some very 
fundamental differences between them, a psychologist, though maybe not an 
aesthetician, could learn much about human creativity from computer models like 
Ani. 

Ani is Not Intuitive 
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Discussion Ani is Not Intuitive 

All depth psychology of whatever school or shade insists on the 
importance of unconscious processes in creativity, and there is no 
question that most of the decisive impulses of the artist— and, indeed, 
of the scientist— -issue, as I mentioned earlier, from below the 
threshold of awareness. This is true, however, for most human 
activities. In weighty as well as in small matters we commonly judge 
and decide on the basis of criteria which we identify only post factum 
and on request, if at all, and which, more often than not, can be 
formulated in sensible speech only with difficulty. At the same time 
we feel quite sure that we did not act arbitrarily or blindly but for 
good reasons.... In a certain sense, then, it is true that in the creative 
process conscious behavior and unconscious behavior are no more 
different from each other than the flowing of a river in full daylight is 
different from its flowing in the darkness of night. 

--Rudolf Arnheim [Arnheim 1962] page 5 



The human mind, because of its intuitive capacity, often jumps over 
several logical steps without realizing it, like Evel Knievel soaring over 
a row of buses. The computer must touch down on every bus, or 
every logical step, in reaching an answer. 

That, of course, is ultimately reassuring. We needn't worry about 
computers getting creative. 

William Bierman. "Getting to Know the People Powered Computer". August 12. 
1976. Chicago Tribune 
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A in is Not Intuitive Discussion 

Do humans soar over a row of buses without realizing it or do they touch 
down on every bus without realizing it? Is intuition the product of skipping steps or 
is it a process that step by step moves forward but one is conscious only of where 
it started and where it ended? If it is the latter, then Ani is only all too intuitive 
being very ignorant of what different parts are doing. My guess is that Ani's 
performance would improve if the opposite were true and Ani could reason more 
introspectively. If intuition is a step-skipping process then parts of Ani might 
qualify too. For example, Ani does not step by step move the characters in a scene 
through their paces to determine where they will end up or how long it will take. 
Instead Ani estimates, makes approximate guesses, and skips thousands of steps. 

The notion that intuition is connected with creativity is a common one. This 
may be because intuitive thought is essentially more efficient because it is not 
slowed down by having to report constantly to some overseeing processes 
(consciousness?). What distinguishes intuitive thought from normal (conscious) 
thought is that the former happens without us being aware of it. There is no need 
to assume that it is different in any other respect. 
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Section B Ani and Aesthetics 
What Aesthetics is 



Aesthetics is no longer essentially a philosophy of beauty, but an 
experimental science based upon psychology, sociology and the theory 
of creativity. One of its basic aims is to find out how creation works 
by studying art objects and how artists made them. 

Abraham A. Moles [Reichardt 1971) page 62 



/ don't think you can get anywhere by trying to define art as this or 

that. You have to talk about varieties of experience which are 

y-^ considered to bear on art, but clearly the word art is a child's word — 

things are either art, or science or art or reality or something, and that 
is a dumb-bell distinction to start with. 

- Marvin Minsky [Chaikin 1979) 



Clearly the putting of literature and music and these things [visual 
arts] into the same boxes is a political and not a psychological 
classification 

-•-Marvin Minsky [Chaikin 1979) 
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What Aesthetics Is Discussion 

Aesthetics is concerned with the nature of the process of creating a work of 
art, the work of art itself, and the process of experiencing a work of art. 1 Ani is a 
working model of the first aspect, the nature of the creative process. This 
research has little relevance to the other two aspects of aesthetics. The place I 
see for a computational view of the nature of a work of art itself is as a way of 
describing the complex structure involved— -how the parts and levels relate and 
interact. However, I think it is misleading to talk as if this structure existed in the 
art object, instead there are descriptions of the art object in the minds of the artist 
or viewer. This distinction is especially important because the structure is not 
isolated but interacts with other knowledge (such as what some part of the art 
/-^ object symbolizes). Insight into the third aspect of aesthetics, the nature of the 

experiencing of works of art, will probably have to wait for computers able to see, 
though a blind computer system that evaluated a high-level symbolic description of 
an art object might prove very enlightening. 

The difference between creating and experiencing a work of art may not be 
as great as is usually believed. Much of the knowledge that is involved in 
constructing a work of art deals with how to make objects that are aesthetically 
pleasing. And much of the aesthetic appreciation of an object involves the 



I. Whilo some of things I will say will apply to live-action film, the static visual arts, and the 
non visual arts, the scope of this discussion is limited to the aesthetic and affective aspects of animation. 
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reconstruction of the mental processes that one supposes occurred in the artist 
while making the object. 

I see the aesthetic quality of a work of art profitably broken into the 
following aspects: 

(a) Structural. This part is concerned with the overall scheme, 
structure, story or script. The aesthetics lies in the relationship of 
the parts and the ways in which the different levels of abstraction 
interact. The top-level structure of Ani's films are provided by 
the user's film description. Below that level, however, Ani 
provides a simple illustration of the reasoning behind the 
construction of such structures, 

(b) Emotional. Ani knows of the moods, feelings, personalities, and 
relationships that motions can convey.' 

(c) Perceptual. This is the part that is concerned with what looks 
pleasant or beautiful. For example, flicker {i.e. alternating dark and 
light frames) for many people is perceptually unpleasant. 2 A more 
positive example is that bright and harmonious colors are typically 
more pleasant than muddy, dull, or clashing colors. This perceptual 
aspect includes more traditional aesthetic concerns such as 
composition and balance. An important component of this is the 
handling of global interactions of the aspects (e.g. between the 



1. An extended Ani would also know how shapes, colors, and the interactions and transformations of 
these different aspects convey affective qualities. 

2. This example, as is the case with most examples in this thesis, is culturally dependent and can be 
considered correct only when all else is normal. The "facts" are easily overridden since any of the other 
aesthetic aspects (structural, emotional or symbolic) or the context can dominate. 
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colors and the shape). Ani currently is very weak in this respect. 
Ideas as to how to improve Ani to take these perceptual aspects 
into account are presented in Subsection 7 "Non-Narrative 
Animation" on page 176. 

(d) Symbolic. The symbolic aspect of a work of art depends upon a 
cultural set of associations between what an image depicts and 
other concepts. I investigated this aspect the least since it 
requires that very large amounts of knowledge of the real world 
be embedded into Ani. 



Ani is an embodiment of a theory of aesthetics that differs from the formal 
theories of mathematicians such as Birkhoff [Birkhoff 1933], Mandelbrot 
[Mandelbrot 1977], Gips and Stiny [Stiny 1979]. They stress the structure and 
/**%, informational content of an art object (and sometimes the information content of 

the process that makes it), while the theory behind Ani stresses the role of 
knowledge and reasoning. Myriad inter-related decisions intelligently made on 
the basis of large amounts of relevant knowledge is the essence of the process 
of artistic creation. Ani is a crude and simple computer model of that process. 
The aesthetic problems of interest by this view are how to structure the 
decision space, how to bring the relevant knowledge to bear for each choice, 
how to handle interactions between the choices of the various parts, between 
the different levels of abstraction, and the different views of the object being 
created, and how to do all this while maintaining a certain coherence and 
consistency. The relationship of this knowledge-oriented view of aesthetics to 
the formal information-oriented theories is discussed in Section B "Aesthetic 
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Systems" on page 133. 

Objections to Ani as a Model of Aesthetics 

I am well aware that this research will raise many objections from aestheticians 
and art theoreticians. A computational model of animation appears to be at 
variance with commonly held views of art as intuitive, imaginative, unpredictable, 
non rule-based, and requiring the creator to have a soul or innate talent. I 
think that an Al view of these problems could be enlightening to the art world 
and should be formulated. Hopefully, Ani will be useful as an example in 
meeting these kinds of objections. 

Many of these objections are similar to those against the idea that Ani is 
creative discussed in Subsection 4 "Objections to the Notion that Ani is 
Creative" on page 209. This is to be expected since Ani is relevant to 
aesthetics primarily as a model of the process of creating a work of art. 
Traditionally people have thought that the role that aesthetics plays in creation 
is unique to the creation of art. Many have come to realize that, on the 
contrary, aesthetics plays an equally important role in creativity in other fields 
such as mathematics and science.' Nonetheless there are a few aspects of 
conventional aesthetics unique to artistic creation that Ani seems to be at 



1. [Wrchsler 197S] is a collection of articles on this very point. See also Subsection 3 "A Digression 
into Aesthetics and Mathematics" on page 53. 
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Objections to Ani as a Model of Aesthetics Discussion 

variance with. New objections to Ani as a model of aesthetic creation arise 
because of the role of perception, emotions, taste and style in aesthetics. Some 
overlap with the previous discussion on creativity is unavoidable, however. 

Lack of Perception 

Ani is perceptually limited to input from a key-board. Ani is blind both to the 
world and her own films. One may wonder how such a perceptually-limited 
system could be expected to produce visual art. A partial answer is that 
perception is a process of creating symbolic structures from external stimulus 
and therefore the system needs only those symbolic structures and not a 
perceptual mechanism. Ani's knowledge base is, to a large extent, a condensed 
and simplified version of the structures that I have built up over the years as a 
perceiver. There are historical cases of great artists or musicians who became 
blind or deaf and yet were able to continue producing great works of art or 
music, Nonetheless, it is quite useful to see what one has produced. Ani 
maintains symbolic structures describing the film and its images as they are 
created. These structures could compensate for most of Ani's perceptual 
deficiencies were critics to evaluate these structures added. (See Section D 
"Critics and Criticism" on page 189 for a discussion of this.) For example, when 
finally running the film Ani easily could be modified to detect that a character 
unexpectedly moved too close to the side of the screen or that a section of the 
film took too much or too little time. 
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This does not completely compensate for a lack of sight however, for there 
are many cases where a creator does not notice some aspect or interaction of 
parts until the product is created and perceived. For example, I once made a 
film that had a very complex shape growing and turning while simultaneously the 
colors were changing in subtle ways. The color changes turned out to be too 
subtle in this context -- I still have troubles convincing people that they are 
really happening. One solution is to exaggerate the aspect that became 
overwhelmed (the color changes) and to downplay the other aspects (the 
rotation and growth speeds). The simplest solution to Ani's lack of feedback is 
to rely on a human observer to give perceptual criticisms. 



f\ Lack of Feelings 
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To call up in oneself a feeling once experienced and having called it up 
in oneself by means of movements, lines, colors, images, expressed in 
words, to so convey this feeling that others experience the same 
feeling — in this consists the action of art... 

■-- Leo Tolstoy [Tolstoy 1 893] page 74 



Painting is a thing of intelligence. One sees it in Manet. Once can see 
the intelligence in each of Manet's brush strokes, and the action of 
intelligence is made visible in the film on Matisse when one watches 
Matisse draw, hesitate, then begin to express his thought with a sure 
stroke. 

Pablo Picasso [Ashton 1977] page 16 
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Lack of Feelings Discussion 

The intricate cybernetic systems which, we may surmise, govern 
thinking in general and creative thinking in particular, must not be 
treated as solely an intellectual affair, or simply a matter of knowing, 
reflecting, analyzing, inferring or engaging in other exclusively cognitive 
activities, in so far as this is possible. The entire individual is 
involved, body and soul. By which I mean that his repertoire of 
heuristic devices is not merely a bag of intellectual tricks and 
shortcuts. Audacity and daring, fearlessness in asking unwelcome 
questions, and willingness to ignore the sign "do not trespass on my 
territory" all these belong to heuristics. 

John Cohen [Reichardt 1971] page 3S 



Some people think that when an artist paints a depressing scene that he or 
she must be feeling depressed while painting it. Many less extreme theories of 
art still have emotions playing a very important role. It might indeed help to be 
feeling in a particular mood to create a consistent and strong visual 
representation of it. Ani need not "feel" to behave the same as people who do. 
It may turn out that feelings are a manifestation of knowledge-based processing 
but regardless, it is not the phenomenoJogy of feelings that is critical here, only 
the effects of the feelings. (See Appendix VIII. "An Essay on Art and Ani" on 
page 291 for a discussion of the expression theory of art as it relates to Ani.) 

Turing's imitation test [Turing 1950] is apropos here. We do not (and 
maybe cannot) know what someone else is feeling, but we can observe and 
interact with people and "feeling" computers and if after a sufficiently long time 
cannot tell them apart then the computers have passed the test. The same test 
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applies to the problems of divining the emotional state of an artist by observing 
his or her art work. At least one such test of a computer program has been 
carried out. Over ten years ago Noll wrote a program that made line drawings 
based upon statistics of a few paintings by Mondriaan [Noil 1967]. One hundred 
people were shown Xerox copies of the computer output and of Mondriaan's 
"Composition with Lines" and 59/100 liked the computer version better and only 
29/100 correctly identified the Mondriaan. 1 

People could be asked whether Ani's films or similar films made by humans 
express the emotions of their maker better. Ani might pass the test which 
would provide evidence against the notion that one needs to feel emotions to 
make art; knowing emotions instead is adequate. 2 Ani does not feel anything, 
has nothing corresponding to emotions inside, nonetheless can animate characters 
that are madly in love, terribly frustrated, angry, frightened, and so on. This is 
because Ani knows what these feelings mean in terms of the gross behavior of 
a character. Ani could even be extended, presumably, to animate as if in a 
particular mood. Ani, extended in such a manner, need not be depressed to 
make a film that looks like one made by a depressed person. The mood would 



1. As pointed out by Luc Steels, this experiment probably misses the point of what Mondriaan was 
doing He was creating the language which his paintings are "written" in and dearly Noll's program 
did not create the language of its drawings. 

2. On the other hand Ani might fail the test not due to an inability to feel but because her knowledge 
of emotions is too crude and that lack of subtlety surfaces in the films. 
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influence decisions, perhaps to prefer darker more somber colors, slower 
movements and exaggerating the unpleasant scenes (such as someone hurting 
someone else). 

Ani Cant Know Enough 

if every stage in the creative process could be logically specified and 
made explicit, we could invite the computer to write poetry and 
discover theorems, for a computer can discharge any task that can be 
formally described. The trouble is that we are not yet in a position, 
either in principle or practice, to set the limits of the formalization of 
mental activities, in spite of over-sanguine assertions to the contrary. 
An enthusiast declares, for example: "Undoubtedly, it will ultimately be 
feasible to program a computer to achieve the same mastery of English 
^f-\ that we ourselves have... Creativity, in every sphere, is a species of 

information-processing in which, so to speak, the processing by man as 
transducer, contributes far more to the output than the information 
itself. The creative man makes more and better bricks with less 
straw. He does not have to plod and search every nook, crevice or 
cranny. He has a knack of eliminating false trails. In short he has a 
repertoire of heuristic devices at his disposal. He himself can tell us 
little about them, and a computer scientist, would give his right arm to 
be able to pounce on them and bring them to light of day. 

John Cohen [Reichardt 1971] page ?6 



A common notion is that the knowledge (or talent) needed to produce good 
art is not explicable or transferable. Art schools just transfer needed technical 
skills but the "creative spark" either is in someone or is not. Or alternatively 
art schools just provide an environment that encourages one to be creative or 
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original without teaching creativity or artistic talent. Many go further and say 
that artistic talent is essentially innate — you either have it or you don't. 

It is very important to know whether there is some truth to this view. If 
it is false then art education could be changed drastically with beneficial results. 
This view of artistic talent, if incorrect, is probably harmful to potential artists 
who believe it. If they think that they themselves have no talent and they 
cannot obtain it, they become defeatist and their self-image often proves 
self-fulfilling. Even if they think they possess talent, their belief that talent is 
essentially innate lessens the chances that they will try as hard to learn to be 
more creative. 1 

Why might some knowledge be unobtainable? It might be embedded within 
some innate processes of someone. For example, it might prove impossible for 
anyone to learn to see red as blue since the knowledge involved is frozen 
forever within some innate vision processes. Or the knowledge might be so 
subtle, so complicated, so difficult to access or so hard to translate into some 
communication language that try as one might one cannot transfer the knowledge 
because one cannot "put it into words". That this is the case can only be 
empirically verified. The record of Al in procedurally explicating knowledge or 



I. Tin? kind of concern with the affects of one's models or theories of one's own mind is developed 
much further in [Minsky draft]. 
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processes that were previously thought to be inexplicable is encouraging, as is 
the success of many teaching experiments conducted by the Logo Group of MIT's 
Al Laboratory ([Papert 1971a] and [Logo 1978], for example). Logo's work on 
teaching people to juggle, ride unicycles, walk on stilts and other "circus" skills 
is especially encouraging. An interesting discussion of knowledge embedded with 
procedures can be found in [diSessa 1977]. 

The construction of systems like Ani are good operational tests of 
questions about the nature of the knowledge needed to make art. A successful 
system would not only provide strong evidence that such knowledge is 
explicable and transferable but might prove to be a good medium for the 
^-s, transfer. Such a system could be used, studied, manipulated and extended by 

art students. This use of a system like Ani is discussed further in Section B 
"Ani and Education" on page 149. 

Everyone Sees Differently 

"What would an art critic says if he were to see it for the first time?" 
I'm not worried about what he would say. That would depend on what 
school he belonged to. Some critics prefer a sea bluer than this, or 
less green, because on looking at the sea they seek the idea which 
they have formed of it through a picture which they have accepted as 
good because they have been told that it is good or because they 
think that the signature of the painter is worth a lot of money. 

Pablo Picasso [Ashton 1977] page 122 
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Another objection to Ani is concerned with the perceptions of the viewer. The 
objection is that each individual perceives a film differently, so it becomes 
problematic as to what model of the viewer the system should have. Much of 
the knowledge of the system consists of facts such as "curving, smooth motion is 
happy", "hate can be conveyed by hitting", etc.. These are culturally-dependent 
generalizations that occasionally fail due to the viewer's idiosyncratic experiences 
and associations. There are at least two retorts to this objection. One is that 
every film or work of art suffers from the same problem and yet there is 
enough of a common ground of associations, perceptions, and experiences for the 
work of art to communicate and affect its viewers. It is upon this common 
ground that Ani's knowledge is based. The other response to this objection is 
to construct idiosyncratic knowledge-based systems as described in Section H 
"Where All This Might Ultimately Lead" on page 199. Each viewer could modify 
the knowledge base according to his or her own feelings and associations and 
then let the system produce the film. A knowledge-based system is capable of 
conforming to each individual viewer's subjective preferences and personality. 

A very limited version of this capability currently exists with Ani. The 
global film description can be viewed as a partial model of the viewer. If the 
viewer is a young child, for example, then the levels of the global descriptors 
can be set so that the film is simple, obvious, flashy, moderately varied, and not 
too long. Perhaps an extended set of such descriptors combined with the ability 
to respond to criticism would provide a mechanism for making films tailored to 



Page 235 



/ ■ 



/""V 



/***v 



Everyone Sees Differently Discussion 

the particular user. 

But the Films Must Be Mechanical 

It is evident that none of the rules of composition are fixed by 
reasonings a priori, or can be esteemed abstract conclusions of the 
understanding, from comparing those habitudes and relations of ideas, 
which are eternal and immutable. Their foundation is the same with 
that of all the practical sciences, experience; nor are they any thing 
but general observations, concerning what has been universally found 
to please in all countries and ages... To check the sallies of the 
imagination, and to reduce every expression to geometrical truth and 
exactness, would be the most contrary to the laws of criticism; 
because it would produce a work, which, by universal experience, has 
been found the most insipid and disagreeable. But though poetry can 
never submit to exact truth, it must be confined by rules of art, 
discovered by the author either by genius or observation. 

- David Hume [Hume 1777] page 269 



It is obvious that we are talking about mechanization of creativity. But 
is this not a contradiction in terms? Almost, but not really. Creativity 
is the essence of that which is not mechanical. Yet every creative act 
is mechanical — it has its explanation no less than a case of the 
hiccups does. The mechanical substrate of creativity may be hidden 
from view, but it exists. Conversely, there is something unmechanical 
in flexible programs, even today. It may not constitute creativity, but 
when programs cease to be transparent to their creators, then the 
approach to creativity has begun. 

Douglas Hofstadter [Hofstadter 1979] page 67? 



f\ 



Page 236 



/"> 



Disru^ioti But the Films Must Be Mechanical 

A common criticism of films or other art objects is that they appear too 
mechanically constructed, that they are too predictable. This is an especially 
common criticism of computer-generated films. The motion is too smooth, it has 
little character. The shapes are too exact, they too lack character. The 
computer films that are exceptions to this are either driven by pseudo-random 
number generators or by humans. The former are often structureless and lack 
coherence since too many decisions were made by "tossing a coin". The latter 
are often fine since they avoid being mechanical by having humans graphically 
communicate the motion or shape to the computer. 

Ani avoids this difficulty by a process somewhat similar to that which a 
/**% human goes through. The motion Ani gives objects typically has character and 

complexity, unless of course the intent is to convey a simple or mechanical 
personality of a character. Basic patterns of motion are combined and modified 
to achieve a very definite character, so most motions are not likely to be 
mistaken as mechanical. Predictability and a "mechanical character" are products 
of simple rules. Ani's rules and the knowledge that they operate with are 
complex and varied enough (and with more effort could be even more so) that 
Ani's films are not predictable and the style is not mechanical. "Mechanical" is 
an adjective that applies to the products of simple rule-based processes but 
need not apply to products of much more complex mechanisms. 
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But the Films Must Be Predictable 

It sometimes seems to be argued that people would become effective 
problem-solvers if only we could teach them to be unconventional. If 
our analysis here is correct, unconventionally may be a necessary 
condition for creativity, but it is certainly not a sufficient condition. If 
unconventionality simply means rejecting some of the heuristics that 
restrict search to a limited subspace, then the effect of 
unconventionality will generally be a return to relatively inefficient 
trial-and-error search in a very much larger space. 

Newell. A.. Shaw, J, and Simon. H. [Newell 1962] page 105 

Some might complain that being deterministic Ani is too predictable. Ani is 
/-\ deterministic in the sense that, unless told to behave differently, will produce 

the same film for the same script. Ani remembers, however, the decisions made 
in making the first film and, if so instructed, can make another that is 
significantly different from the first. Ani could be extended to inject an 
unexpected or unpredictable element into a film without the use of random 
number generators. Ani could establish a pattern and then break it at a critical 
point. Ani could decide to use one aspect of a character as an odd 
distinguishing feature. For example, an improved Ani might give a very pleasant 
friendly character a large pointed shape (typically an unfriendly shape). The 
motion, color, line, and texture would override the undesirable connotations of 
the large size and pointed shape. The atypical size and shape for the friendly 
character might be chosen because the character is supposed to be a strong 
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friendly ally. This treatment is consistent with one of the major themes of this 
research --- that Ani should have a good reason for every choice made. 

Section C Ani and Style 

When the system is determining the speed, path, shape, color and 
texture of each object, its task would be greatly simplified if each 
aspect were properly structured. Intuitive spaces are desirable, so 
are standardized ones. The way the space is structured or described 
greatly influences the ease of maneuvering and reasoning within it. 

Ken Kahn in the thesis proposal for this research [Kahn 1977b] 

^ I% ^ Critics often talk about the style of an artist, a group of artists, or a 

period but it is obviously a slippery quality. It refers to something in common 
among a group of art objects which distinguishes them from other objects. 
What makes it so slippery is that any aspect or combination of aspects can be 
used to separate styles. Art objects can differ in content, technique, structure, 
purpose, or affect. The differences can be superficial aspects (such as the kind 
of brush stroke) or conceptual. 

What does style mean in the context of computer systems such as Ani 9 It 
has something to do with the decision space that they function within and the 
way they move within that space. Ani does not structure the decision space for 
the dynamics of a character. It is broken down into four aspects (movement, 
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attraction/avoidance tendencies, path, and characterization).' Each aspect in turn 
is broken down in elements such as speed, avoidance of strangers, path 
curvature, and deliberateness. If this space were structured differently, a 
different style would result. Another example is the choice of colors. If Ani 
were extended to determine the colors of the characters, then some color space 
would be used to make choices within. Different styles of coloring would result 
depending upon whether the system worked within a red-green-blue space, a 
hue-intensity-saturation space, or a "semantic net of colors". 

It is not just the structure of the space that influences style, but the way 
one makes choices within it. Any regularities in the process are potentially 
perceivable as part of a style. For example, a system that completely 
determined the description of each character one at a time would have a 
different style from one which determined the colors of the characters, then 
their dynamics, then their shapes and so on. 2 

Style is a result of the regularities in a decision-making process that are 
not inherent. If the regularity were a necessity then it would not be part of 
the style since every such object would be regular in that way. Style can only 
be perceived against a background of alternatives. A well-designed system 



1. This is described in more detail in Section A "What is a Value?" on page 103. 

2. Ani can explore the spaces of choices in either order. Experiments to determine the different 
results have vet to be made. 
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could be an ideal experimental medium for seeing how the same task is handled 
as parameters that control the style are manipulated. The global film descriptors 
in Ani could be used to this end. When one says that the film should be subtle, 
simple, varied, original and not very flashy, one is partially defining a style within 
which Ani will operate. These parameters do not control the decision space 
much but they do influence in a systematic way how choices within it will be 
made. An interesting problem for future research is how the style itself could 
be determined. Is there a meta-aesthetics of artistic styles? 

Section D Ani Seen from Many Perspectives 

Ani is a computer program that can be profitably described from several 
different perspectives. From a functional point of view, Ani is a very naive 
computer animator capable of making simple animated sequences in response to 
vague and incomplete descriptions. From the point of view of artificial 
intelligence, Ani is a program that demonstrates the usefulness of some concepts 
such as actors, suggestions, postponement, levels of abstraction, and explicitly 
controlled and recorded reasoning. From the same point of view, Ani is an 
example of a way of making aesthetic choices in under-constrained situations. 
From an Al engineering point of view, Ani is a complex knowledge-oriented 
computer program that is built out of parts that perform functions such as 
gathering suggestions, choosing what choice to think about next, choosing values 
and methods, noticing and resolving conflicts, and planning sequences of actions. . 
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From the point, of view of epistemological engineering, Ani represents knowledge 
of emotions, aesthetics, interpersonal relationships, and animation that, while 
admittedly in a manner that is too limited and ad hoc, extends the kinds of 
knowledge currently representable in a machine. To psychologists interested in 
understanding creativity, Ani is a computational embodiment of a theory of 
creativity. Similarly, of relevance to art theorists, the framework upon which Ani 
was constructed is a computational model of the production of aesthetic objects. 
To researchers in computer graphics, Ani is an illustration of the usefulness of 
representing complex knowledge in the computer in order to generate images. 
To the animation, or more generally the entertainment industry, Ani is a small 
first step towards the automatic production of custom-tailored entertainment. 
From an educational point of view, Ani is a repository, in an explicit and precise 
form, of some of the knowledge and techniques needed to produce animation. 

Finally, as a doctoral thesis, a description from any one of these viewpoints 
alone gives a very distorted view of Ani. Each viewpoint by itself is an 
inadequate description of Ani, and so I have attempted to describe Ani from 
many different perspectives. The focus of all these perspectives of Ani 
concerns the importance of basing decisions in a sensible way upon as much 
relevant knowledge as possible. The thesis is that creation is a 
knowledge-intensive computational process. It is this aspect of Ani — in the 
world of programs that create aesthetic objects — that makes her interesting. 
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Ani Seen from Many Perspectives 



Animation is the one medium which can deal graphically with the 
fantastic world of the future. There's no limit to what we can do. 

-Walt Disney [Thomas 1958] page 179 
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Appendix I - The Cinderella Film Description 

This appendix presents the entire description of the Cinderella film as it was 
presented to Ani. The four portions of a film description (the character, the 
character relationship, the scene, and the overall film descriptions) are presented. 
Film descriptions are the only part of Ani that a user needs to know about. The 
representation of the descriptors, choice points and so on are of interest only to 
one who desires to modify or understand Ani's knowledge. 

Upper case is used in what follows to signify words that have meaning to Ani 

(or Director) independent of a particular story or knowledge base. 

(DEFINE Cinderella CHARACTER ;; Cinderella is a character who 
(PROCESS INITIAL DESCRIPTION 

(PHYSICAL (AND beautiful shabby)) ;; is both beautiful and shabby 
(PERSONALITY (AND good friendly hard-working shy)) ;; is good, friendly 
(ROLE-IN-STORY MOST-IMPORTANT))) ;; has the most important role in the story 

(DEFINE st.opmot.her CHARACTER ;; stepmother is a character who 
(PROCESS INITIAL DESCRIPTION 

(PHYSICAL ugly) ;; is ugly 

(PERSONALITY (AND mean selfish strong evil)))) ;; and mean selfish ... 

Relationships are broken into two parts, how the first character relates to 
the second and how the second relates to the first. For example, the relationships 
of Cinderella and her stepmother are described by 
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The Cinderella Film Description 

(DFTTNF. (RELATIONSHIP-OF stepmother Cinderella) RELATIONSHIP 
(PROCESS INITIAL DESCRIPTION ;; the stepmother relates to Cinderella by 
(AND dominates hates))) ;; dominating and hating her 

(DEFINE (RELATIONSHIP-OF Cinderella stepmother) RELATIONSHIP 
(PROCESS INITIAL DESCRIPTION ;; while to her stepmother Cinderella is 

(AND is-obedient-to is-tolerant-of )) ) ;; tolerant and obedient 

The other characters and relationships are defined as follows. 

(DEFINE fairy-godmother CHARACTER ;; the fairy godmother is a character who 
(PROCESS INITIAL DESCRIPTION 

(PHYSICAL (AND pretty magical)) ;; is magical and pretty 
(PERSONALITY (AND good kind strong)))) ;; and good, kind and strong 

(DEFINE (RELATIONSHIP-OF Cinderella fairy-godmother) RELATIONSHIP 
; ; Cinderella's relationship with the fairy godmother is 
(PROCESS INITIAL DESCRIPTION 
/""""N ; ; that she is polite and grateful to her 

(AND is-polite-to is-grateful-to))) 

(DEFINE (RELATIONSHIP-OF fairy-godmother Cinderella) RELATIONSHIP 
; ; and the fair v godmother relates to Cinderella by being 
(PROCESS INITIAL DESCRIPTION 

; ; protective, generous, and helpful 
(AND is-protective-of is-generous-towards is-helpful-towards) ) ) 

(DEFINE prince CHARACTER ;; the Prince is a character who 
(PROCESS INITIAL DESCRIPTION 

(PHYSICAL (AND beautiful strong)) ;; is strong, handsome. 
(PERSONALITY (AND good stubborn determined))));; good, stubborn... 

(DEFINE (RELATIONSHIP-OF Cinderella prince) RELATIONSHIP 
(PROCESS INITIAL DESCRIPTION ;; Cinderella loves the Prince throughout 
loves) ) 
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(DEFINE (RFI ATIONSIIIP-OF prince Cinderella) RELATIONSHIP 
(PROCESS INITIAL DESCRIPTION ;; And the Prince loves Cinderella 
loves ) ) 

The first scene in the film introduces Cinderella and her stepmother, their emotional 
states, and their relationship as follows. 

(DEFINE introduction SCENE ;; introduction is the name of this scene 
(PROCESS INITIAL DESCRIPTION 
(AND ;; m any order do the following 
(ESTABLISH (PERSONALITY Cinderella)) ;; establish Cinderella's personality 
(ESTABLISH (EMOTIONAL-STATE Cinderella ;; and that she is mildly happy 

(JOY (POSITIVE LOW)))) 
(ESTABLISH (PERSONALITY stepmother)) ;; now for the stepmother 
(ESTABLISH (EMOTIONAL-STATE stepmother (JOY (POSITIVE LOW)))) 
(ESTABLISH (RELATIONSHIP ;; and establish 

(RELATIONSHIP-OF stepmother Cinderella))) 
/*"N ; ; how stepmother relates to Cinderella 

(ESTABLISH (RELATIONSHIP ;; and how 

(RELATIONSHIP-OF Cinderella stepmother))))) 
; ; Cinderella relates to her stepmother 
(SET YOUR LENGTH TO LONG)) ;; and this scene should be relatively long 

The second scene is more typical in its mixture of event types and their partial 

ordering. 
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(DEFINE kcpt-apart SCENE ;; this scene is called "kept apart" 
(PROCESS INITIAL DESCRIPTION 
(SEQUENCE ;; and in the following order 
(CONVEY (wants Cinderella ;; convey that Cinderella wants to 

(meets Cinderella prince))) ;; meet the Prince 
(CONVEY (prevents stepmother ; ; then convey that the stepmother prevents 

(meets Cinderella prince))) ;; Cinderella from meeting him 
(AND ;; and then in any order or simultaneously 
(ESTABLISH (EMOTIONAL-STATE Cinderella ;; establish that Cinderella is 

(JOY (NEGATIVE HIGH)))) ;; very unhappy 
(ESTABLISH (EMOTIONAL-STATE stepmother ;; while her stepmother is 

(joy (positive medium))))))) ;; happy 
(SET YOUR LENGTH TO LONG)) ;; this scene should be relatively long 

The other scenes were defined as follows. 

(DEETNE no-longer-kept-apart SCENE ;; this is called "no longer kept apart" 
(PROCESS INITIAL DESCRIPTION 

(CONVEY (undoes ;; convey that the fairy godmother undoes 
fairy-godmother 

(FROM-SCENE kept-apart ;; whatever in the "kept apart" scene 
(CONVEY ;; was used to convey that 
(prevents stepmother ;; the stepmother prevents 
; ; Cinderella from meeting the Prince 
(meets Cinderella prince))))))) 
(SET YOUR LENGTH TO SHORT)) ;; this should be a short scene 
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(DEFINE mooting SCENF ;; the fourth scene is called "meeting" 
(PROCESS INITIAL DESCRIPTION 

(SEQUENCE ;; in the following order 
(CONVEY (alone Cinderella prince)) 
; ; convey that Cinderella and the Prince are alone 

(CONVEY (getting-it-on Cinderella prince)) ;; then they "get it on" 
(AND ;; and then establish their relationships 
(ESTABLISH (RELATIONSHIP (RELATIONSHIP-OF prince Cinderella))) 
(ESTABLISH (RELATIONSHIP (RELATIONSHIP-OF Cinderella prince)))) 
(AND ;; and then establish that they are both very happy 
(ESTABLISH (EMOTIONAL-STATE prince (JOY (POSITIVE HIGH)))) 
(ESTABLISH (EMOTIONAL-STATE Cinderella (JOY (POSITIVE HIGH))))))) 
(SET YOUR LENGTH TO LONG)) ;; this should be a long scene 

(DEFINE justice SCENE ;; the "justice" scene is defined as follows 
(PROCESS INITIAL DESCRIPTION 
(AND ; ; in any order 
.^ ; ; convey that the stepmother is aware of the Prince and Cinderella 

(CONVFY (aware stepmother (getting-it-on prince Cinderella))) 
; ; and that they are "getting it on" 
(CONVEY (getting-it-on prince Cinderella)) 

(ESTABLISH (EMOTIONAL-STATE stepmother (JOY (NEGATIVE HIGH)))))) 
; ; my] that the stepmother is very unhappy 
(SET YOUR LENGTH TO MEDIUM)) ;; this should be an average length scene 



The global film description and focus follow. 
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(DEFINE cinclorcl ln-f i lm FILM ;; the film is called Cinderella film and should be 
(SET YOUR LEVEL OF variety TO MEDIUM) ;; moderately varied 
(SET YOUR LEVEL OF complexity TO LOW) ;; pretty simple 
(SET YOUR LEVEL OF originality TO LOW) ;; not very original 
(SET YOUR LEVEL OF coherence TO HIGH) ;; very coherent 
(SET YOUR LEVEL OF obviousness TO HIGH) ;; not at all subtle 
(SET YOUR LEVEL OF flash iness TO LOW) ;; not very flashy 
(SET YOUR LEVEL OF energy TO HIGH) ;; very energetic 

(SET YOUR FOCUS TO the-f i lms-focus) ;; and its focus is called the films focus 
(SET YOUR FILM-LENGTH 

TO (INTERVAL (MINUTES Z) (MINUTES 5)))) ;; 2 to 5 minutes long 

(DEFINE the-fi lms-focus FOCUS ;; and the focus is defined as follows 
(SET YOUR ORDFR-OF-CIIARACTER+ELEMENT-TYPE+ELEMENT TO 

(CHARACTERS ELEMENT-TYPES ELEMENTS)) 
; ; the coherence of characters has the highest priority 
(SET YOUR PARTS-OF-THE-FOCUS TO 
j#«V ; ; the scenes, characters, and relationships to emphasize are 

(Cinderella ;; Cinderella 
kcpt-apart ; ; the scene in which she is kept from meeting the Prince 
no- longer-kept-apart ;; and the scene in which she meets to Prince 
(RELATIONSHIP-OF Cinderella stepmother) 
; ; how Cinderella relates to her stepmother 
(RELATIONSHIP-OF stepmother Cinderella)))) 
; ; and how her stepmother relates to her 
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Appendix II - The Running of the Cinderella Film 

This appendix describes how the detailed descriptions of films produced by Ani are 
run, i.e. turned into an animated film. Director, the programming language that both 
Ani and Ani's films are implemented in, was extended to be able to simulate the 
behavior of the characters and coordinate several parallel activities. The interface 
between Ani and Director is discussed especially with regard to how it converts 
symbolic descriptions of movement and location into numerical values for display 
commands. 

Ani does not really animate anything, but instead produces a detailed 
f*S description of what should happen and Director, a different computer system [Kahn 

1978b], takes that description and animates it by making images upon a computer 
display. The distinction between the two systems is due to the extreme 
differences in their degrees of generality and intelligence. Ani is very specialized, 
making only with a narrow kind of non-figurative dramatic animation in a distinctive 
style. Director is a general-purpose programming language with many graphical and 
animation capabilities. Ani makes many decisions using high-level knowledge, 
Director makes no decisions of any consequence and just does what it is told. This 
is because the film description by the time it gets to Director, has very little 
ambiguity or incompleteness left. Also, the quantities that Ani deals with are 
primarily symbolic (e.g. "the upper far left corner of the screen" or "a high speed") 
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while Director deals exclusively with ordinary numbers (e.g. 3.14159 or 16). 

Section A The Simulation of Ani's Characters 

The characters that star in Ani's films move in accordance with a complex 
description of their dynamics. They interact with each other (e.g. chase, accompany, 
or hit) and often move until some condition is satisfied (e.g. "in contact with 
Cinderella", "ten seconds have passed"). To implement these abilities Director had 
to be extended. These extensions use heavily Director's mechanism for 
quasi-parallelism which is based upon the notion of a quantum of time called a "tick" 
[Kahn 1 978d]. Upon every tick of an imaginary clock each actor does a tick's worth 
of activity. A tick is similar to a film frame — - it is important that at the end of the 
frame all the actors on the screen are arranged correctly but anything can happen, 
in any order, between frames. 

For example, the objects of Director had to be extended to "walk" in 
accordance with a character's motion, path, style, and tendency descriptions. Any 
moving character has a motion description consisting of a speed and acceleration. 
They are used to determine how much the character should move on each tick. The 
acceleration was to be used to get the character up to speed and the speed is 



Page - 251 






/"■s 



f\ 



j0^^*>, 



The Simulation of Ani's Characters Running of Cinderella Film 

proportional to the distance traveled on a tick. 1 A character couid just turn 
towards its goal and then move the distance to be traveled (unless it is so close to 
its goal that this would move it past its destination) but this would be inadequate 
for conveying the personality and mood of a character, Since a character should 
rarely change its orientation instantaneously, turning around is also controlled by the 
speed (and acceleration) of the character. The path description for a character is 
important in giving the character's movements some character. It consists of the 
path curvature (since characters typically move in a series of arcs), the length of 
each jaunt (the length of each arc) and the degree of smoothness of its movements. 
This enables Ani to describe a character's movements (either in general or for a 
particular activity) to be short and jerky, long and graceful, or slow and deliberate. 

Moving to a location is even more complicated. Sometimes a character should 
move by avoiding any contact with others, sometimes it should contact others only if 
forced to (i.e. if cornered) and even then it might depend upon its relationship with 
the other characters, while other times the character should definitely come in 
contact with the other (for pushing and hitting for example). Usually a character 
avoids going off the screen, but occasionally this is just what is called for. 



1. Implementing the acceleration correctly turned out to be harder than expected and was temporarily 
removed (i.e. the acceleration is instantaneous). The difficulty is in the decelerations where the program 
must determine when to begin to start slowing a character down so that it ends up at the right place at 
the right time despite the fact that its target may be moving. 
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Punning of Cmripiella Film The Simulation of Ani's Characters 

Sometimes the movement of a group of characters should be locked together so 
that when, for example, a character is accompanying another, they do not both use 
their own dynamics since then they would probably drift apart and go their own 
ways. All of these abilities had to be built into the routines for moving characters. 

To stop a character after it has started to move Ani needs the ability to 
exercise many different options. Sometimes a character should stop when it 
reaches its goal, sometimes when it has traveled the original distance to its goal 
even if the goal has moved in the meanwhile, sometimes a character should move, 
for a certain length of time, and other times it should move until some other activity 
begins or ends. This expressive power was added to Director by associating with 
each moving character an actor that is its "stop condition". Upon every tick the 
stop conditions are told of the current distance to the goal and asked if they should 
stop their respective owners. 

Section B Running Activities in Parallel 

Ani describes each activity to occur in a film by indicating the characters involved, 
their actions (e.g. walking, hitting, etc.), and the appropriate dynamics for each 
character. The characters are run in parallel by sending each a tick message asking 
them to do a tick's worth of action. This is not enough for running Ani's films 
however. The activities themselves must be coordinated. Director was extended 
to permit activities to be repeated, to be run either sequentially or concurrently, 
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and for the beginning and end of activities to signal the initiation or termination of 

other activities. For example, in the second part of the second scene the Prince 

wanders across the screen while Cinderella and her stepmother are performing the 

"standing guard" routine. Cinderella moves towards the Prince and her stepmother 

quickly intervenes by blocking her. Cinderella then moves back a bit and her 

stepmother soon follows. This is repeated several times. The top-level 

coordination of these activities is described by Ani as follows, 

(and ;; do the following concurrently 
(activity-of make-guarding-more-lnteresting 1) ;; make the Prince wander around 
(repeat 6 ; ; the following is repeated 6 times 

(and ;; the following are started together 
(activity-of prisoner-escapes 1) ;; Cinderella moves towards the Prince 
(activity-of guard-chases 1) ;; after waiting the step mother pursues 
(activity-of prisoner-gives-up 1) ;; after being blocked Cinderella retreats 
(activity-of resume-guard 1)))) ;; and soon after the stepmother follows 



The activity called "guard-chases" is typical and can be paraphrased as 
follows. 1 This activity is to start a quarter of a second after the beginning of the 
"prisoner-escapes" activity (the number 1 in its name indicates that this is the first 
use of this activity in the film). First the dynamics of the stepmother are initialized 
so that she will move about 147, faster than what is usual for this entire section of 
the scene (the "stand guard" method normally sets the speed of the "guard" to 607. 



I. For those curious about what it really looks like see Appendix VI. "Some Code Written by Ani" on 
page 272. 
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faster than normal for this part but the subscene was stretched and so the increase 
was cut down). The usual dynamics for the stepmother in this section indicate that 
her speed should be fast, that she move very directly and deliberately, that her 
path be only moderately smooth, that she should accelerate quickly, and should not 
avoid contact with Cinderella. The first movement is the stepmother turning to face 
a location that is a small way from Cinderella in the direction of her goal. 1 This 
heading is offset for the small amount of curvature in her motion. The stepmother 
then moves to a moving goal that she just faced towards (the exact location of the 
goal is recomputed on every tick). When she gets there not only does the activity 
end but it sends a message to the "prisoner-escapes" activity asking it to stop. 

f*\ Section C The Interface between Ani and Director 

Ani communicates with Director by giving it descriptions of the subscenes which 
contain the activity and character dynamics descriptions. 2 One problem is that Ani 
rarely uses ordinary numbers but instead uses symbolic descriptions of quantities 



£*****, 



1. Cinderella's goal in the "prisoner escapes" activity (which is running concurrently) is the Prince. 
However, because the "standing' guard" method works in general for any prisoner being held captive at 
some location and since Cinderella is being kept in a place that is "away from the Prince" when she 
escapes Ii<t goal is really represented as a place that is away from a place that is away from the Prince 
which is simplified as towards the Prince. 

2. Due to address space limitations Ani and Director do not fit together in the computer's address 
space (256k) and therefore Ani really communicates with Director by writing out files describing each 
scene and a Director extended as described reads and executes those files. 
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and locations such as "a fast speed" and "a small way from Cinderella and just off 
the edge of the screen" while Director accepts only ordinary numbers (such as 
1.618). My objections against Ani using ordinary numbers are partly 
phenomenlogical — they don't feel right -— and partly computational — much 
economy can be gained by restricting the system to a small set of numbers such as 
"a very little bit", "a little bit", "a little", "a medium amount", "a lot", "an awful lot". 
The advantages of such numbers are that they permit assertions to be made about 
the numbers themselves (such as consistently using the number "an awful lot" will 
result in a wild, confusing movie), they enable crude cross-aspect comparisons ("a 
little bit" of motion will "correspond" somewhat to "a little bit" of size), and their 
context-dependent meaning simplifies Ani's reasoning. The last point needs some 
clarification. The meaning of the "upper left corner" or "a high speed" depend upon 
the particular dimensions of the display screen and many constants describing the 
current norm (e.g. that a normal character takes about eight seconds to cross the 
screen). This effectively separates Ani's reasoning from the particulars of the 
display. Also the operations upon these approximate numbers can themselves be 
approximate. 1 



1. This tisp of symbolic numbers is similar to fuzzy arithmetic [Zadeh 1978]. Its use in another Ai 
system is discussed at length in [Kahn 1975a]. 
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Running of Cinderella Film The Interface between Ani and Director 

The convenience of what are sometimes called "qualitative numbers" [Minsky 
draft] has as its price the necessity of a mechanism for converting qualitative 
numbers to ordinary numbers. Ani does not need to do this — reasoning just fine 
with only qualitative numbers — it is for communicating with Director that the 
conversion is necessary. The expression "a long typical path iength" is converted 
into numbers by finding out that a typical path length is 5'/. of the size of the screen 
and that "long" means double the usual. Each element has its typical value and each 
of the qualitative numbers corresponds to a multiplicative factor. 1 

More complex is the conversion of descriptions of a position on the screen to 
real coordinates. Location descriptions can be either symbolic screen coordinates 
such as "the upper far left" or relative descriptions such as "a small distance from 
location- 1 and in the direction of location-2". Of course to evaluate a relative 
location something needs to be fixed (usually as a screen position). The conversion 
to coordinates is done locally by asking the actor for the location (in this case "a 
small distance from") for the real coordinates. It in turn asks location- 1 and 
location-2 for their real coordinates. If they are at least half the screen apart then 
it picks the location that is one fifth of the way from location- 1 on the line between 
thern. If they are closer it picks a spot a third of the distance between then. I 



I. In other contexts when two qualitative lengths are added, for example, then the qualitative numbers 
correspond to different numbers. 
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apologize for the arbitrariness of this and discuss in Section A "Making Ani Do 
More" on page 162 and Section D "Critics and Criticism" on page 189 improvements 
to Ani so that locations were chosen more intelligently. 
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Appendix III - Time and Memory Statistics 

Many people are curious about how large a program Ani is, how long it takes to 
make a film, what kind of machinery is used and the like. This appendix attempts to 
satisfy such curiosity. 

Ani and Director were implemented upon a time-shared PDP-10 <KA 
processor) at MIT's Al Laboratory. The displays used were Knight TVs (a black and 
white frame buffer of about a quarter of a million points) and a color frame buffer 
of the same resolution with 4 bits per pixel permitting at most 16 different colors 
at any one time selected from over two billion possible colors. 

Ani is too large for the address space of the computer (256K) to be run 
along with complete Director, so they are run separately and communicate by 
writing files for each other. Ani alone cannot fit into memory when making the 
entire Cinderella film and so is run in two passes. The first pass generates the 
descriptions of the dynamics of the characters. The part of Ani needed to do this 
grows to about 180K. The second pass reads a file summarizing the results of the 
first pass and generates the activities and subscenes that are passed to Director to 
be run. This part is 1 76K (24K of which are free storage) and grows to over 21 OK. 
In terms of actors (which unlike more self-consistent actor systems are only the 
larger entities in Director) this pass begins with 418 actors and creates 154 new 
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ones (not including temporary ones). 



Director (with all its graphical primitives loaded) is 11 7K (7K free storage) 
and grows by over 14K when extended to run Ani's movies. The programs grows 
very largo as the frames of the scenes are generated and stored as actors. For 
example, after making the introduction scene of the Cinderella film at 4 frames per 
second it was over 196K. 

The source programs (admittedly a crude measure of size) involved in the 
making and projection of the Cinderella film total 125 IK blocks. 



/•"N 



(a) 32 blocks are for the basic part of Director. 

(b) 14 are to support the graphical and animation capabilities of 
Director (this does not include the 47 blocks of source code that 
supports the TV turtle [Goldstein 1975]). 

(c) 4 are for the efficient compilation of actors and message 
transmissions into Lisp. 

(d) 66 blocks are for Ani herself. 

(e) 15 (of which 6 are also part of Ani) are need to extend Director 
to run Ani's films. 
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For communication between the different passes while making the Cinderella 
film the first pass of Ani wrote a 3 block long file to the second pass. The 
second pass wrote five files totaling 28.5 blocks of programs for Director to 
run. Director wrote 29 blocks to describe the films in Lisp for the Lisp 
compiler to compile. (Director does display the movie, however, a great speed 
up results from the compilation of the movies and their being run as compiled 
Lisp.) 

Not only are the programs large but they take a while to run. All the Lisp 
programs are compiled as is most of the Director code but efficiency of the 
system had a low priority in this research.' Ani is a prototype system intended 
only to demonstrate the feasibility of the embodied concepts and could be much 
f* more efficient. Nonetheless it is interesting to see how much time was taken 

and where. The first pass took over 7 cpu minutes to compute the dynamics of 
the four characters in the film. This involved 54 decisions for the relative 
values of the dynamic elements and 48 for absolute values. Over 300 actors 
were involved in the process. The results of this pass are communicated to the 
next pass by writing a file in 30 cpu seconds. The second pass took about 2.5 
cpu minutes to determine the activities of the characters and about 2 cpu 
minutes to write the files to communicate this to Director. Director took 19 



I. Most of the Director code is translated into Lisp with most of the message passing overhead 
compiled out. 
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minutes to execute the program written by Ani. This included the simulation of 
the characters, checking for collisions, and moving characters in a complicated 
manner in accordance with their dynamics. It then took 9 cpu minutes to display 
the frames and to compile the movie into Lisp code. The Lisp compiler then 
took about 4.5 minutes to compile the movie. To project the compiled movie of 
over 400 frames took 3 cpu minutes. 

All together a complete run of Ani takes about 48 cpu minutes. Another 
20 minutes is consumed by Ani writing out files describing and justifying the 
choices made. When debugging the system it typically runs about half speed in 
order to catch and localize bugs. Combine this state of affairs with a 
time-shared computer that depending upon the load multiplies the compute time 
by a factor of up to 5 or 6 times (larger factors occur but I don't run Ani then) 
and the fact that three jobs each about 200K are needed. Machine limitations 
of this kind have prevented Ani from being tested much on variations of the 
Cinderella film and from being easily extended to do more. A Lisp machine 
([Bawden 1977] and [Weinreb 1979]) might cut the cpu time down by a factor 
of 2 or 3 (partly because the whole system could reside in memory at once 
obviating the need for writing long files to communicate) and the elapsed time 
by a factor of 10 (partially because it is a single-user machine). Ani could be 
optimized to run at least twice as fast. Research along the lines of Ani could 
proceed on such a machine since it would take it less than ten minutes to make 
a simple movie-— ten minutes to make a movie that would take an animator a 
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few days. 

Another machine limitation that hindered this research is that only crude 
previews of the movies can be seen prior to putting it on film. Much of the 
Cinderella movie, despite its simplicity, cannot be projected faster than 3 or 4 
frames per second and even then the timing is thrown off by the garbage 
collector and the time sharing system. As a result it was difficult to fine tune 
Ani ? s display knowledge. An implementor needs visual feedback, for example, to 
decide how much faster one character should be than another during a chase or 
how long it should take for a character to turn around. The crudeness of Ani's 
display methods affects the ease with which a viewer can perceive the intended 
activities and thereby the quality of the films. This problem would alleviated by 
the proper hardware (either a fast enough single-user computer or else a means 
of storing each frame as video and playing it back). 
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Appendix IV - Descriptors as They Look in Ani 

This appendix presents a sample of the different descriptors as they are defined in 

Ani. The details (such as what the back-quote "'" mean) are not important, the 

descriptors are presented mainly for those curious about what they really look. 

What is of interest is that they are all actors with their own databases and are 

capable of answering a variety of questions and requests. 

(PROCESS SUGGESTIONS ;; whose suggestions follow 

(FOR SPEED LOW STRENGTH LOW) ;; weakly suggests low speed 

(FOR ATTRACTION-TOWARDS-ENEMIES (NEGATIVE MEDIUM) STRENGTH HIGH) 

; ; strongly suggests avoidance of enemies 

(FOR ATTRACTI0N-T0WARD5-STRANGERS (NEGATIVE HIGH) STRENGTH HIGH) 

; ; strongly suggests strong tendency to avoid strangers 

(FOR ATTRACTION-TOWARDS-FRIENDS (POSITIVE LOW) STRENGTH LOW) 

; ; weakly suggests a slight attraction towards friends 

(FOR (ESTABLISH 'character shy) 

; ; one method for establishing a character as shy is 
'(METHOD: TYPE CONVEY ;; to convey that 
VALUE (BIND ((other 

; ; some other character who is a stranger 
(CHARACTER-THAT-SATISFIES 
(stranger-of .character NIL)))) 
(avoids .character N other))) 
; ; is avoided by the character 

STRENGTH 'MEDIUM)) ;; this is moderately strongly suggested 
(ADD brazen TO 
YOUR LIST OF OPPOSITES) ;; the opposite of shy is brazen 

(ADD forward TO YOUR LIST OF 
OPPOSITES)) ;; and forward 
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(DEFINE hates AFFECTION-RELATIONSHIP-DESCRIPTOR 
; ; h.itfs is a (negative) affection relation 
(PROCESS SUGGESTIONS ;; it suggests that 

(FOR (ESTABLISH (hates ?character ?other-character ) ) 
; ; to establish that a character hates another 
(OR ; ; either of the following suggestions are appropriate 
'(METHOD: TYPE CONVEY ;; convey that 

VALUE (avoids .character .other-character) 
; ; the character avoids the other 
BEST-WHEN ((NOT (dominates .character 

.other -character)))) 
; ; this is best when the character doesn't dominate the other 
STRENGTH 'MEDIUM ; ; and is moderately suggested 
'(METHOD: TYPE CONVEY ;; another thing to convey would be that 
VALUE (hurts .character .other-character) 
; ; the character hurts the other and 

BEST-WHEN ((dominates .character .other-character))) 
; ; and this is best when the character dominates the other 
STRENGTH 'HIGH ; ; this is strongly suggested 
'(METHOD: TYPE CONVEY ;; or convey that 

VALUE (pushes .character .other-character) 
; ; that the character pushes the other 

BEST-WHEN ((dominates .character .other-character))) 
; ; which is best when the character dominates the other 
STRENGTH 'LOW))) ; ; this is only weakly suggested 
(ADD loves TO YOUR LIST OF OPPOSITES) ;; love and hate are opposites 
(ADD despises TO YOUR LIST OF NEIGHBORS)) ;; despises is similar to hates 
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(DEFINE (variety level) GLOBAL-DESCRIPTOR 
; ; tho variety level n a global film descriptor 

(DO WHEN RECEIVING (YIELD SUGGESTIONS FOR ?choice-point) 

; ; which when asked for suggestions 
(LET ((element-name (ASK .choice-point RECALL YOUR ELEMENT-NAME) ) ) 
; ; place the name of the element in question in the variable element name 
(LET ((possible-values 

(ASK .element-name RECALL YOUR POSSIBLE-VALUES))) 
; ; find out from that element name what are the permissible values 
(AND possible-values ;; if there are any then 

; ; pick a value based on the desired level of variety, the element's 
; ; possible values, and the previously chosen values for the element 
'(SUGGESTION: ELEMENT ,(ASK .choice-point 

RECALL YOUR ELEMENT) 
VALUE .(PICK-VALUE-WITH-VARIETY 

(ASK :SELF RECALL YOUR LEVEL) 
possible-values 
?^ element-name) 

STRENGTH LOW ; ; this is only weakly suggested 
SOURCE , :SELF)))))) ;; the source is really an 

; ; instance of this actor 
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(DEFINE meets moves ;; meeting is a special cnse of moving 
(ADD PREREQUISITE FOR (meets ? ?b) (on-stage ,b)) 
; ; our of the prerequisites for meeting someone is that they be on stage 
(ADD PREREQUISITE FOR (meets ?a ?b) 

(prerequisites-of (moves ,a (vicinity-of ,b)))) 
; ; another prerequisite is that one can move close to the other 
(DO WHEN RECEIVING (YIELD YOUR FIRST PART OF (meets ?a ?b)) 
; ; when asked about what is the first part of meeting someone reply 
'(METHOD: TYPE DISPLAY ;; the displaying of 

VALUE (move-smal 1-part-towards ,a (vicinity-of ,b)))) 
; ; the character moving a small part of the way to the other 
(DO WHEN RECEIVING 

(YIELD SUGGESTIONS FOR {and ?task (convey (meets ?who ?other))>) 
; ; when asked how to convey that someone meets another 
'((SUGGESTION: ELEMENT ,task ;; suggest that one 
VALUE (METHOD: TYPE DISPLAY 

; ; display the character moving to the other 
/"S VALUE (move-to ,who (vicinity-of , other))) 

STRENGTH HIGH ; ; this is highly suggested 
SOURCE meets))))) 
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(OFF INF good PFRSONAI ITY-CHARACTER-DESCRIPTOR 
; ; good describes a character's personality 
(PROCESS SUGGESTIONS ;; and suggests the following 

(FOR (ESTABLISH ?character good) ;; to establish that a character is good 
(OR ; ; try either 
'(METHOD: TYPE CONVEY ;; conveying that 
VALUE (BIND 

((other ;; some other who 
(CHARACTER-THAT-SATISFIES 
; ; is not the same character and 
; ; is preferably weak and good 
(IS-NOT , character 

(IS-DESCRIBEO-AS (weak good)))))) 
; ; is helped by the character 
; ; who we are trying to establish as good 
(helps .character N other))) 
STRENGTH 'HIGH ;; this is strongly suggested 
/•"> '(METHOD: TYPE CONVEY ;; or else convey that 

VALUE (BIND ((bad-thing 

(EVENT-THAT-SATISFIES ;; some event 
( is-considered-bad-by .character))) 
; ; considered bad by the character 
(prevents .character v bad-thing) ) ) ) 
; ; is prevented by that character 
STRENGTH 'HIGH)))) ;; this too is strongly suggested 

The second suggested method has a bug in it as pointed out by Jon Doyie. The 
event should be considered bad by more then the character involved. Even evil 
people try to prevent events they think bad (for them). 
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Appendix V - Representation of Suggestions 

This appendix presents a few examples of suggestions as they are represented 
within Ani. The three major types presented are suggestions for absolute values, 
suggestions for relative values, and suggestions for methods or activities. 

The format of a suggestion is 

(SUGGESTION: ELEMENT <element-description> 
VALUE < value-description) 
STRENGTH <High, Medium, or Low) 
SOURCE <pointers back to the originators of the suggestion)) 

Examples of the three major types of suggestions follow. 

A suggestion for the value of an atomic element is 

(SUGGESTION: 
ELEMENT speed ; ; This suggestion is for the typical speed 

; ; It is for the stepmother because this is located in her database 
VALUE high 

STRENGTH medium ;; this is the average strength of the suggestions from each source 
SOURCE (and ;; and is compatible with the suggestions of the following 

(relative-choice-point-of (stepmother Cinderella) relative-speed) 

; ; the relative speed of Cinderella and her stepmother (defined below) 

(relative-choice-point-of (stepmother fairy-godmother) relative-speed) 

strong ;; and "strong", a character descriptor 

(energy level) ;; and "energy level", a film descriptor 

(variety level) 

(flashiness level))) 
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A suggestion for a relative value is 

(SUGGESTION: 
ELEMENT (relative-speed Cinderella stepmother) 

VALUE le.ss-t.hfin ;; This suggests that Cinderella be slower than the stepmother 
STRENGTH medium 

SOURCE (and ;; and is compatible with the suggestions of the following 
dominates 

(UNIQUENESS-SUMMARY: UNIQUENESS-LEVEL high 

DESCRIPTOR-TYPE all 
OF-WHO Cinderella) 
; ; the uniqueness of Cinderella also helped suggest this 
... ; ; oilier uniqueness summaries helped suggest this also 
(UNTQIIENESS-SUHHARY: UNIQUENESS-LEVEL high 

DESCRIPTOR-TYPE all 
OF-WHO stepmother) 
; ; as did the uniqueness of the stepmother 
/""N (UNIQUENESS-SUMMARY: UNIQUENESS-LEVEL high 

DESCRIPTOR-TYPE personality 
OF-WHO stepmother) 
(COMPARISON-SUMMARY: SIMILARITY-LEVEL (negative medium) 

; ; they were found to be rather different 
DESCRIPTOR-TYPE all 

OF-WHO (comparison-of Cinderella stepmother)) 
; ; the comparison of the two contributed to this suggestion 
. . . ) ) ; ; other comparison summaries helped suggest this 
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A typical suggestion for what to do to establish or convey something is the 

following for conveying someone is preventing another from meeting a third by 

removing the second one's ability to move near the other. 

(SUGGESTION: 
ELEMENT ( in-order-to (convey (prevents stepmother 

(meets Cinderella prince))) 
; ; this a suggestion for how to convey that the stepmother prevents 
; ; Cinderella from meeting the Prince by conveying that 
(convey (undoes stepmother 

(can-move Cinderella 

(vicinity-of prince))))) 
; ; the stepmother makes Cinderella unable to move towards the Prince 
VALUE (METHOD: 

; ; the prerequisites for this method are 
PREREQUISITES ((can-get-to 
stepmother 
(away-from (vicinity-of Cinderella) 

(vicinity-of prince))) 
; ; that the stepmother can get between Cinderella and the Prince 
(faster stepmother 

Cinderella)) ;; and she is faster than Cinderella 
BEST-WHEN ((dominates stepmother ;; best to use this when the 

Cinderella)) ;; stepmother dominates Cinderella 
TYPE display ;; this is a display type of method 
VALUE (stands-guard 

stepmother ; ; the suggested method is that the stepmother 
Cinderella ;; stand guard over Cinderella by keeping her 
(away-from (vicinity-of Cinderella) ;; away from the prince 
(vicinity-of prince)))) 
STRENGTH high ;; this is strongly suggested 
SOURCE undoes) ;; by the Undoes actor 



/*""% 



Page - 271 



f~S 



Some Code Written by Am 



Appendix VI - Some Code Written by Ani 



f\ 



The following is one small sample of the program that Ani wrote to make the 

Cinderella film, A few inconsequential parts were removed but otherwise the 

following is exactly as it was written by Ani and read by Director. The 

capitalization of Director constants and the comments were added. 

(DEFINE (subscene-of kept-apart 1) director-subscerte 
; ; this is the first subscene of the kept apart scene 
(SET YOUR scene TO kept-apart) ;; is part of the kept apart scene 
(SET YOUR stretched-by TO 1.0) ;; did not happen to be stretched or shrunk at all 
(SET YOUR transition-type TO cut) ;; a film cut is called for here 
(SET YOUR (location-of Cinderella) TO 

; ; Cinderella should start out at the following location 
(place: where (screen-position upper far-right) 

; ; the upper part of the far right part of the screen 
why (so that (move-smal 1-part-towards Cinderella prince)) 
; ; this was chosen so that Cinderella and the Prince were far apart 
status (placed by (subscene-of kept-apart 1)))) 
(SET YOUR (location-of prince) TO 

; ; the Prince is in this scene though far away from Cinderella 
(place: where (good-distance-from (vicinity-of Cinderella)) 

why (so that (move-small-part-towards Cinderella prince)) 
status (placed by (subscene-of kept-apart 1)))) 
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(SET YOUR ( location-of fairy-godmother) TO 

(place: where nowhere ;; fairy godmother is not on stage 
why (cut because 

; ; because a cut was called for because Cinderella 
; ; and the Prince were left too close to each other 
; ; by the previous scene 
(known- to-be -distance 
(distance-between (vicinity-of Cinderella) 
(vicinity-of prince) 
(subscene-of kept-apart 1)) 
small )) 
status undone)) 
(SET YOUR (location-of stepmother) TO 

(place: where nowhere ;; the situation with the stepmother is the same 
why (cut because 

(known -to-be-distance 
/""""n (distance-between (vicinity-of Cinderella) 

(vicinity-of prince) 
(subscene-of kept-apart 1)) 
small)) 
status undone)) 
(SET YOUR plans TO ( (activity-of moving-small -part-towards 1))) 
; ; this is the only activity planned for this subscene and is defined below 
; ; if more than one is planned then their ordering would have been given 
(SET YOUR running-time TO 1.53)) ; ; estimated to take about a second and half 
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(DEFINE: (act.ivit.y-of moving-smal 1-part-towards 1) plan 
; ; thi< i; the first (mid only) planned activity of this type in this scene 
(SET YOUR begin-time TO 0.0) ;; and it should begin right away 

(SET YOUR subscene TO (subscene-of kept-apart 1)) ;; is part of the first subscene 
(SET YOUR plans TO 
((sequence: ;; do the following step by step 

(ASK Cinderella SET YOUR special-dynamics TO 

; ; provide Cinderella with her dynamics special to this activity 

| (special-dynamics-of Cinderella in (subscene-of kept-apart 1 ) ) -26 ) ) 
(ASK Cinderella INITIALIZE YOUR CURVATURE-SIGN 

; ; She will not move in a straight line but wiggle back and forth. 
; ; This declares the way that she will wiggle first 
TOWARDS (small -way- from (vicinity-of Cinderella) 

(vicinity-of prince))) 
(ASK (activity-of moving-small-part-towards 1) 
CAUSE (Cinderella) TO GRADUALLY GO IN ARC 
f 00 **, ; ; gradually make Cinderella face towards and then move, 

careful OF COLLISIONS TO ;; avoiding any collisions, to 
( small -way-from (vicinity-of Cinderella) ;; a place a small way from 
(vicinity-of prince)) ;; Cinderella towards the Prince 
AS moving-target ;; and if he moves keep updating your destination 
STOPPING IF 

; ; Stop after covering the current distance to the goal 
; ; In this case, it is the same as when goal is reached 
(move-only-original-distance '(subscene-of kept-apart 1))))))) 
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(DEFINE (special-dynamics-of Cinderella in (subscene-of kept-apart 1)) 
special-dynamics 
; ; thr default dynamics of Cinderella in this segment of the scene are 
(SET YOUR path-jaggedness TO medium) ;; move in a moderately jagged path 
(SET YOUR path-curvature TO low) ;; with little curvature 

(SET YOUR path-length TO medium) ;; and each curve should be moderately long 
(SET YOUR del iherateness TO high) ;; moving very deliberately 
(SFT YOUR repetiveness TO medium) ;; and somewhat repetitively 
(SET YOUR acceleration TO high) ;; quickly getting up to speed 
(SET YOUR rhythmic-level TO high) ;; in a highly rhythmic manner 
(SET YOUR activity-level TO high) ;; and very active in general 
(SET YOUR speed TO high)) ;; and at a high speed 

(OFFTUT | (special-dynamics-of Cinderella in (subscene-of kept-apart 1))-?6| 
; ; this i< wlirie modifications to Cinderella's dynamics in the first subscene are kept 

(special-dynamics-of Cinderella in (subscene-of kept-apart 1)) 
; ; and it inherits the defaults from the sub-scene's dynamics of Cinderella above 
(SF1 YOUR (speed factor) TO 2.0)) ;; multiply her speed by 2.0 
; ; it turns out this was the only modification to her normal dynamics 

A few of the details require explanation. Most of the values are still 
symbolic and yet Director needs numeric values for parameters such as the speed, 
location, and path length. These symbols are translated into numbers but by an 
interface that is loaded into both Ani and Director. The translation of expressions 
such as "upper far right", "small way from the vicinity of Cinderella in the direction 
of the Prince" and "high acceleration" into numerical values in done in manner that is 
sensitive to dimensions of the display and parameters for the normal amount of time 
to cross the display or to turn around completely. This is described in greater 
detail in Appendix II. "The Running of the Cinderella Film" on page 250. 
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Some of the information presented above is not strictly necessary for making 
the film. It would have been used if Ani were able to redo a scene after running it. 
For example, the running time estimate could be used to redo the scene if it took 
much more or less time than estimated. Similarly the reason for the cut is recorded 
so that if some other change causes the Prince and Cinderella to start off more than 
a small distance apart the cut will not happen. 

Currently, the deliberateness, repetiveness, rhythmic-level, activity-level, 
and acceleration are not used by the extended version of Director that runs this 
code. All but the acceleration and deliberateness, however, are used in making 
choices. 
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Appendix VII - A Knowledgeable Diagram Maker 

A simple example that illustrates the knowledge-oriented approach to computer 
graphics is a diagram making program called DIAGRAMER that produced many of the 
diagrams in this report. To Director one can describe text, boxes, arrows, labels 
and so on and it will display them. 1 The difficulty with this way of making diagrams 
is that one must describe so much detail — the coordinates, the dimensions, and 
shapes of the each parts. Descriptions at such a low-level are difficult to debug 
and modify because they are long, do not fit a user's way of conceiving of the 
problem, and because the data is very interdependent (e.g. if a box moves so 
^^ should the arrows pointing to it). The inconvenience of this can be alleviated by a 

good interactive graphical interface in which one just points to where objects 
should go by moving a mouse or pen connected to a digital tablet. 2 A graphical 
interface combined with some means of constraining what is to be connected to 



1. Director is (he computer graphics language that both Ani and DIAGRAMER are written in. It is 
described in [Kahn 1976b] and [Kahn 1978b]. 

2 These are among the many devices which could be used that communicate the movements of one's 
hand to the computer. 
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what is a very good means of producing and modifying diagrams. 3 A very early 
example of a system with these capabilities is Sketchpad [Sutherland 1963]. The 
state of the art of such systems is probably Borning's Thinglab [Borning 1979]. 

One still has to declare many constraints to the computer and give more 
commands than one would like. For example, suppose one wants a box with some 
text with in it. Why can't one just say that to a computer and let it determine 
what size character font to use, where to place the box, how big the box should be, 
where the text should be within the box, and so on? The idea behind DIAGRAMER 
is that as long as any of the computer's "decisions" can be overridden, the machine 
should do this. DIAGRAMER is designed to let one describe to a computer what one 
wants in the diagram. Sketchpad and its descendants enables one to graphically 
indicate what the diagram should look like and to describe how it should be 
constrained. 



jS******V 



?. Certain modifications are difficult to communicate graphically either because they are global 
changes (eg. changing the dimensions for the entire diagram) or because the change necessitates the 
change of other parts that depend upon the part being changed. Describing these relationships and 
constraints is possible, but if such constraints are given to a sufficiently intelligent system then one need 
not describe it graphically at all. Another limitation of graphically controlled systems of this sort is that 
they cannot be used by other programs. Diagrams are an excellent medium for communication: why 
should not a program use diagrams to explain itself? 
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The simplest form of "knowledge" that DIAGRAMER needs to contain are 
default values. 1 Unless a user indicates otherwise, DIAGRAMER uses its default 
font, for example. One could have default dimensions for a box, but the default 
would seldom be acceptable since it can contain any amount of text. What is 
needed is a procedural default for the dimensions that takes into account the 
amount of text and the size of the font. DIAGRAMER has these and so to define a 
box we only type the following. 
(DEFINE ani box) ;; This states that "Am" is the name and text of a box to be displayed 
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Figure 1 A box drawn by DIAGRAMER 
Suppose we want another box on the screen that says, "Director". The problem is 



1. Defaults are such a simple form of knowledge that I hesitate to use that word for them. 
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that the default position is the same as the "Ani" box and they will be drawn on top 
of each other. Clearly DIAGRAMER should be able to place them better and indeed 
it can. DIAGRAMER is always working on a default diagram and as we type 
(DEFINE director box) ;; make another box and call it "director" 

a new box named "Director" is added to the default diagram. DIAGRAMER has a 
few experts for different numbers of boxes in a diagram and as this box is added it 
switches from the "one box expert" to the "two box expert". This expert places 
the boxes according to whether the drawing area is horizontal or vertical as 
follows. 
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Figure 2 Two boxes drawn in a horizontal space 
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Figure 3 Two boxes drawn in a vertical space 

This expert is also responsible for deciding which box goes where. In this situation 

it puts the larger box on top (if the diagram is primarily vertical) or to the left (if it 
is horizontal). 

Boxes alone do not make very informative diagrams. "Links" are needed and 

are added as follows 

(LINK ani |written in| director) 
; ; create a link from ani to director labeled "written in" 
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Figure 4 Two boxes and one link 
Notice that the "ani" and "director" boxes have changed places. This is because the 

"two box" expert puts the box with the most links coming out of it in its favored 

position (left or top depending upon the diagram dimensions). Many decisions had to 

be made to add this link. They are so obvious that most people do not notice them. 

For example, DIAGRAMER picked the center of the right side of the "ani" box and 

the center of the left side of the "director" box for the beginning and end of the 

arrow. It placed the text horizontally in the center of the arrow. It used a smaller 

font for the text on the label and so on. Notice how the diagram changes if a user 

changes its dimensions to make it vertical. 
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Figure 5 Two boxes and one link in a vertical diagram 
Let us add another link from "ani" to "director" to illustrate how DIAGRAMER 

automatically rearranges the existing diagram. 

(LINK ani Iprograms i n j director) 
; ; define another link and label it with "programs in" 
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Figure 6 Two boxes and two links in a horizontal diagram 
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Figure 7 Two boxes and two links in a vertical diagram 
Notice the difference between these two diagrams. And yet they differ only by the 

dimensions of the space. In the vertical one DIAGRAMER puts the labels at 

one-third and two-thirds of the distance between the boxes so that they do not 

conflict. This is not necessary for the horizontal one so that default of placing the 

label in the center holds. Notice too that the first link moved over to make room 

for the second link. 
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The complexity increases as we move to more boxes and more links. This is 
well illustrated by the "four box" expert. Let us create two more boxes and a few 
more links as follows. 

(link director J tf seel for| |Computer Animation! ) 
; ; create a link from "director" to a new box with "Computer Animation" in it 

(link director |used for| (Education!) 
; ; create another link from "director" to a new box labeled with "Education" 

(link ani | makes! | Computer Animation!) ;; make another link from "ani" 

After a short while DIAGRAMER draws the following. 
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Figure 8 A Finished Diagram by DIAGRAMER 
The "four box" expert tries to put the box with the most links (the "director" box 

in this case) in the upper left and those boxes with many links to or from it as near 

as possible (which is why the "ani" box is in the lower left). Notice that 

DIAGRAMER's heuristic for where to link the arrow could have done a better job in 
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the link between "director" and "computer animation". It would look nicer if it 
went to the top of the "computer animation" box. Also there is much empty space 
between the "education" and the "computer animation" box which need not be 
there. These kinds of problems point not to any fundamental problem with a 
knowledge-oriented diagram maker, only much more work on DIAGRAMER needs to 
be done. 

Until DIAGRAMER can perform better, a way for it to accept user commands 
or advice is essential. The boxes are extensions of Director "performers" which 
are generalizations of Logo turtles [Goldstein 1975]. They can be told to turn, go 
forward or back, or move to some coordinates. Boxes can be moved by the "Instant 
Turtle" mode in Director which responds instantly to single key-strokes which is 
very convenient if graphical input devices such as a mouse or tablet are lacking. 
Just moving the boxes is not enough since DIAGRAMER must be able to draw the 
modified diagram. For example, to fill the empty space above the "computer 
animation" box, we can type the following. 
(ASK | Computer Animationl HOVE UP 300) 
This results in the following diagram. 
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Figure 9 DIAGRAMER redrawing a diagram after user modification 

Note that the arrows were changed in accord with the move. This is a consequence 

of the fact that the actors involved know upon who else- they depend. Links know 
which boxes they link and boxes know which links come and go from it. This way 
as a box is moved the links can be automatically updated. 

Higher-level modifications are also possible such as exchanging two boxes as 
follows. 
(ASK ani TRADE PLACES WITH IComputer Animation)) ;; they take each other's location 
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Figure 10 The diagram after two boxes are switched 

Notice how the "director" and "ani" boxes have been enlarged automatically to 

create more room for all the arrows coming in or out of their sides. 

I hope that the style of programming graphics in a "knowledge-oriented" 
manner has been illustrated by DIAGRAMER. DIAGRAMER needs to be extended to 
be generally useful. It can detect and fix links that cross each other. It notices 
(and corrects) boxes that are too close to each other only in the case where there 
is not enough room for the text on the arrow linking them. For larger diagrams with 
many boxes there needs to be a way to group boxes together, place the boxes 
within a group, place the groups and then fix up any conflicts. Shapes other than 
rectangles should be available and DIAGRAMER would need to know how to place 
and link them. DIAGRAMER should try to fill space more evenly (currently it does 
for boxes but not links). These deficiencies are not fundamental, with some more 
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effort they could all be remedied. 1 DIAGRAMER is very convenient to use because 
a user communicates with it at a much higher-level than is normal. This is the 
result of placing a small amount of relevant knowledge into the computer. 

DIAGRAMER is currently being completely redesigned to work in a manner 
much more like Ani. Suggestions for where to place boxes, how big they should be, 
which fonts should be used, and the like will be generated, combined and followed. 
Difficult choices will be postponed and the ordering of decisions will be guided by a 
focus. This redesign will be described in a later document. 



I. DIACR A MER represents about If. as much effort to build as Ani and about 4% as much code. 
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Appendix VIII - An Essay on Art and Ani 



In any event, if computers do get good at artistic craftsmanship, there 
may be some question about wherein the art lies. If a person writes a 
program capable of 10,000 string quartets a day, do those quartets 
constitute the art? Probably not. Oddly, but inevitably, the art will be 
in the program. 

Patrick Winston [Winston 1977] p. 254 



This appendix deals with the implications of a successful extension of Ani. 
Suppose all the technical difficulties have been resolved and the system has 
demonstrated its ability to produce a variety of excellent films in a versatile 
manner. Suppose it has passed a limited variation of Turing's famous test of 
mechanical intelligence. When the films are shown to experts and critics who know 
not how they were made, they agree the films are as good as those made by human 
animators. 

I shall discuss what the existence of such a system would suggest or imply 
about questions in aesthetics. Particular attention will be placed upon theories of 
art as expression.' Readers who strongly doubt that Ani could ever be extended 



I. The expression theory of art is just one among many theories of art. Essays like this one about the 
formalist theory of art. the imitation theory and others would presumably also be worthwhile. 
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sufficiently to pass this limited version of the Turing test should try to put aside 
their prejudices and see if this discussion might shed some light upon these 
questions. The advantage of considering these questions with respect to intelligent 
machines is that we know how they work and reason to a much greater extent than 
with human artists. 

Section A Expression Theory 



/-% 



Several theories of art require that the works of art express something of the 
feelings, emotions, ideas, thoughts, attitudes, viewpoints and preferences of the 
artist.' These feelings are communicated from the artist to the viewer via the work 
of art. The different variations of this view are about the relationships between 
the feelings and the artist, the work of art, and the viewer. Before I discuss the 
applicability of this view of art to works of art produced by intelligent machines, we 
need to determine what the work of art is, who the artist is, and whose feelings are 
involved. 



f^. 



I In the interests of brevity I will use the term "feelings" to cover the entire list of aspects that are 
expressed in works of art. 
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Is It Art? 

The question of whether the products of the system are entitled to be called "Art" 
is two questions combined: What is Art? and Does the product match the definition 
of art? If one takes the point of view of a sociologist that art is that which society 
considers art, then under the assumption of this essay that experts could not 
distinguish the system's films from those of human artists (using the computer as a 
medium, just as the system does) then the system produces art. The question of 
whether it is art if art necessarily entails the expression of human emotions and 
feelings will be discussed later. 

Who is the Artist? 

Assuming for the moment the question of whether the product of the system is art 
is settled, a rather difficult question arises— who is the artist? Typically the 
person who makes the work of art is the artist of that work. Interpreting the word 
"person" as broadly, the computer system itself should be considered the artist. 

There are many cases of works of art, however, that were not made by the 
artist. Some contemporary sculptors design a work of art and then have workers in 
a factory construct it. One usually credits these sculptors with creating the work of 
art even if they never touched it. No one seriously would call the factory workers 
that made the object the artists. Architects often operate in this fashion, however, 
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in this case it is perhaps better to consider the design as the work of art and not 
the building itself. This reasoning, however, leads one to consider the plan for a 
painting to be the art object not the painting (especially when it was not painted by 
the artist.) 

Analogous to the situation with these sculptors and architects, one might 
argue that it is the person who presents the script to the machine that should be 
called the artist and the machine is simply the most complex tool ever used by an 
artist. Consider, however, the film description for the Cinderella film. The 
conversion of the fairy tale into a script is fairly straight forward and probably 
could be done by anyone with some understanding of the computer system. Of 
course, he or she might, instead of borrowing from a fairy tale, conceive of a 
wonderful script, ideally suited for the medium, but would not such a case be better 
described as a collaboration between a human script writer and a computer 
animator with both considered the artists? 

It may be a confusion to consider either the computer system or the script 
provider as the artists of the products of the system. The artist may be the person 
that provided the computer with the rules and knowledge that enabled it to' 
produce the film. The programmer of a computer film is normally considered the 
artist, why should this case be different? The distinguishing metric is the extent to 
which the program makes intelligent decisions and choices. At one extreme the 
program might contain instructions on how to make a hundred excellent films and it 
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simply picks the one closest to the script. Here we would want to credit the 
programmer that put those one hundred films into the computer. At the other 
extreme, no one wants to call the teacher of the maker of a work of art the artist 
of that art object. The system under discussion lies somewhere between these 
two extremes. 

If one were determined to give credit to the creator of the system one might 
take the point of view that the computer system itself is a work of art. Its 
programmer is the artist, not of the films, but of the intelligent work of art that 
produces more conventional works of art. This view is more sensible than it might 
appear at first if one considers whose feelings, emotions, and attitudes are 
f"\ expressed by the system. The films are not what should be displayed in art 

museums and galleries but the computer itself. The computer system which 
"viewers" present scripts and see the resulting films is the work of art. The 
person who placed his or her aesthetic knowledge and feelings into the machine is 
the artist. 

In summary, the ascription of the term "artist" depends upon which aspect of 
the creative act is primary. If one considers the artist of a work of art to be the 
agent that through an intensional act is the one who is primarily responsible for 
existence of that work of art, then the script giver is the artist. If one accredits 
the intelligence that makes the majority of important aesthetic choices (such as 
which colors, forms, rhythms, textures, and harmonies are used) as the artist, then 
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the computer system is the artist. If one equates the artist with the source of the 
feelings, emotions, attitudes and preferences expressed in the work of art, then the 
programmer of the computer system is the artist. 

This question arises less in the case of human artists since typically they 
perform all three functions. Collaborative or group efforts are often difficult to 
analyze in these terms because of the complexity of the divisions of labor. It is 
only in the case of computer intelligence that these functions can break into parts 
so smoothly. 

What is the Art Object? 

r% The common view is that the films produced by the computer system are the art 

objects. A strong contender for that privilege is the fully expanded description of a 
film in terms of primitive commands to a computer display. Given the film 
description and a description of the primitives of the display, any sufficiently large 
and fast computer connected to an appropriate display can produce the film. The 
description is analogous to the celluloid of a standard film which can be projected 
by any compatible movie projector. The description can be projected upon a 
display by any compatible computer system. 
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But if we accept this reasoning for considering the description as the work of 
art (along with the necessary hardware to "project" it) then why not state that the 
original film description and intelligent computer animating system in combination are 
the work of art since any reasonable computer can take the two and produce and 
display the "film"? Ignoring the question of the determinism and motivation of human 
artists, this is equivalent to saying that a work of art is the sum of ideas, an artist, 
and a means of realizing the ideas. This may seem strange until one reflects that 
this is a very reasonable view of the art object in the case of the performing arts. 
A play, for example, is the sum of a script, actors, and a means of performing it. 

This computational view of the nature of art objects provides some insight 
into questions of uniqueness, originality, reproduction, and forgery. The ability to 
make perfect reproductions with absolutely no information loss is a consequence of 
the work of art being represented digitally. Forgery and originality become nearly 
meaningless concepts when absolutely perfect copies are easily created. This is 
already the case with literature; the computer may extend it to the visual arts and 
musical performance. Replacing these problems are the new problems of dealing 
with slight variations of a work of art. Is a it a new work of art even though it 
differs from another work of art in only one aspect? What is special about this 
problem when dealing with a computer is its ability to vary any element or aspect 
independently. The color, tempo, viewpoint, or any individual element can be 
changed and a new product painlessly results. 
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Whose Emotions and Feelings? 

The emotions and feelings expressed by the work of computer art come from the 
computer system that made it. The question is whether the feelings are the 
computers or it is just passing them along from either the script writer or the 
programmer of the computer. This question is related to the previous one of who 
is the artist or creative force that made the work of art— related but not identical. 
It would be perfectly consistent for someone to maintain that the computer program 
is the artist that creates the films and yet claim that the feelings and emotions 
expressed are not those of the program but of the script giver or the programmer 
or both. The films are the result of a collaboration between the programmer whose 
f^ feelings and attitudes are represented in the program and the script writer whose 

feelings are conveyed in his or her script. 

It is interesting to speculate how one would answer these questions if the 
system were extended to create its own scripts. To simply say that the 
programmer of the system's inner self was being expressed through the films via 
the program is inadequate. What if the system produced a film that the 
programmer clearly did not intend, that does not accurately express the 
programmer's inner self? Would we say that the system was insincere, that there 
must be a mistake in the program, or that this situation cannot happen? 
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What makes this situation interesting is that many people claim that only 
when the computer "on its own" makes art should we credit the computer instead 
of the human controlling it. The computer is a deterministic mechanism that follows 
the instructions in a program provided by humans. The instructions might tell it to 
learn from its environment, to read several books, but some people still bothered 
by the fact that its learning processes and its reading processes are provided by 
the programmer. One might be lead to think it impossible for a computer to do 
anything "on its own" and therefore it cannot make art. But are humans so 
different? Though intelligent computers with the complexity and performance 
comparable to humans presently do not exist, let us suppose they could be built. 
Then we must either throw out the concept behind the expression "on its own" or 
else give it an interpretation in terms of different levels of description. 

The same sort of reasoning applies to the question of whether computers will 
be able to feel emotions. If people can do it so can computers. The question of 
whether it will be their feelings and emotions or those of their creator is no 
different from the previous question of whether the decisions the computer makes 
are its own or its programmer. 

If there are no differences, then why talk about computers at all one might 
wonder. One consequence of considering the question from the point of view of 
intelligent machines is that it suggests that emotions need not be felt in order to 
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behave in the same manner as one who does feel those emotions. 1 Consider the 
effects of "feeling emotions". Decisions are altered, processes are initiated, and 
others are inhibited. These effects can be created in a machine without feeling 
those emotions. 

Again the advantage of considering the computer when trying to understand 
questions in the expression theory of art is that is it easier to unpack concepts that 
normally only occur as a whole. With respect to feelings, we can now sensibly talk 
of feeling emotions and the effects of feeling emotions as separate components. 
Whether both are necessary to create art is a question worthy of thought, though I 
see no reason why the effects of feeling are not sufficient. It is only the effects of 
jf\ feelings that are perceived by the viewer. 

Section B Summing Up 

What this excursion into the consequences of intelligent art-producing machines has 
yielded is the different components of concepts such as artist, work of art, and 
feelings. When considering single feeling artists producing unreproducible works of 
art, one tends to think of these concepts as unitary. This essay has attempted to 



1. The question of whether emotions ever need to be felt to induce behavior or whether it is the 
behavior (including physiological changes) that create the sensation of feeling. William James was 
concerned about this and decided in favor of the latter. 
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show that at least three different senses of artist are usually lumped together, that 
different levels of abstraction are applicable when considering what is the art 
object, and that it is profitable to separate feeling emotions from its effects. 
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